Download Project DoCTA: Design and use of Collaborative Telelearning

Project DoCTA:
Design and use of
Collaborative
Telelearning
Artefacts
Barbara Wasson
Frode Guribye
Anders Mørch
with contributions from
Eskil Andreassen Konrad Morgan
Rune Baggetun Glenn Munkvold
Arnstein Eidsmo Trond Pedersen
Knut Ekker Kurt Rysjedal
Hege Higraff Helge Underhaug
Øyvind Meistad Jo Wake
Pedagogisk informasjonsvitenskap
Universitetet i Bergen
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Skriftserie for
Forsknings- og kompetansenettverk for IT i utdanning (ITU).
www.itu.uio.no
Produsert i samarbeid med Unipub forlag
ISBN 82-7947-005-0
ISSN 1500-7707
© 2000 ITU
Det må ikke kopieres fra denne boka i strid med åndsverkloven eller
med avtaler om kopiering inngått med KOPINOR, interesseorgan for
rettighetshavere til åndsverk.
Omslag: Nordahl & Jøntvedt
Sats og trykk: GCS Multicommunication AS
Unipub forlag er en avdeling i Akademika AS,
som eies av Studentsamskipnaden i Oslo
2
Preface
Om ITUs skriftserie
Forsknings- og kompetansenettverk for IT i utdanning (ITU) ble opprettet som en del av KUFs handlingsplan om ’IT i norsk utdanning 199699’, og er nå blitt videreført for en ny 4-års periode under Handlingsplanen ’IKT i norsk utdanning, Plan for 2000 – 2003’.
ITUs hovedaktivitet er å igangsette forsknings- og utviklingsprosjekter
innen feltet IKT og utdanning. Blant andre aktiviteter skal den også fungere som en nettverksnode mellom ulike forskningsmiljøer i Norge.
ITU har gjennom perioden 1997-99 fokusert på begrepene læring og
kommunikasjon innenfor skjæringspunktet av teknologi, pedagogikk og
organisasjon. Det har vært et omfattende fokus på teknologiens rolle
som katalysator for endring innen det tradisjonelle utdanningssystemet.
Vi har i dag en rekke erfaringer knyttet til ulike endringsperspektiver,
men trenger fortsatt å systematisere og klargjøre hva endringene består
i. Eksempelvis trenger vi både kvalitative og kvantitative innsyn knyttet
til teknologi i læringssituasjonene. Dette vil være viktige satsningsområder
for den nye perioden.
Skriftserien omhandler ulike typer tekster som har til felles at de tar opp
utfordrende perspektiver relatert til IKT og utdanning. Det gjelder
utredningsarbeid, prosjektrapporter og artikkelsamlinger. ITU har, gjennom skriftserien, som siktemål å bidra til systematisk kunnskap om IKT
og utdanning, samt å skape debatt og refleksjon om de utfordringer vi
står overfor.
Vi håper med andre ord at skriftserien kan bidra til å presentere nye
perspektiver på fremtidens utdanningssystem.
ITU, februar 2000.
3
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Utgivelser i denne serien:
Rapport nr. 1
Ola Erstad Innovasjon eler tradisjon
Rapport nr. 2
Sten R. Ludvigsen mfl. Elektronisk ransel: Ny
teknologi – nye praksisformer
Rapport nr. 3
Svein Østerud mfl. Når ideer flyter sammen …
Rapport nr. 4
Mona Hovland Jakobsen Skoleveien videre
Rapport nr. 5
Barbara Wasson mfl. Project DoCTA: Design and
use of Collaborative Telelearning Artefacts
Rapport nr. 6
Geir Haugsbakk Interaktivitet, teknologi og læring
Rapport nr. 7
Harald Haugen m.fl. SULDAL
Rapport nr. 8
Sten R. Ludvigsen og Svein Østerud (red.) Ny
teknologi – nye praksisformer. Teoretiske og
empiriske analyser av IKT i bruk
Rapport nr. 9
Ingeborg Krange mfl. Samarbeidsorientert læring i
skolen med distribuert bruk av interaktiv 3D - en
evaluering av erfaringer fra prosjektet EduAction
4
Preface
Table of Contents
1 Introduction ........................................................................................................ 17
1.1 Research Questions ................................................................................. 18
1.2 Highlights ................................................................................................... 21
1.3 Organisation of the Report ..................................................................... 23
2 Theoretical foundations .................................................................................. 29
2.1 Computer supported collaborative learning ...................................... 29
2.2 Genuine interdependence ..................................................................... 32
2.3 Collaborative telelearning ...................................................................... 33
2.4 Coordination theory ................................................................................. 33
2.5 Awareness ................................................................................................. 37
2.6 Distributed collaborative learning communities ............................... 38
3 Conceptual framework for evaluations ........................................................ 41
3.1 Vygotsky and Cultural-Historical Psychology .................................... 42
Internalisation ........................................................................................... 43
The mediating role of artefacts ............................................................ 43
Zone of proximal development .............................................................. 45
3.2 Basic principles of Activity Theory ...................................................... 45
Unit of analysis .......................................................................................... 46
The activity system .................................................................................. 46
Object-orientedness ................................................................................ 49
Hierarchical structure of activity ......................................................... 50
Internalisation/Externalisation ............................................................. 51
Artefacts and mediation ......................................................................... 52
Development .............................................................................................. 52
3.3 Methodological Issues ........................................................................... 54
3.4 Some general remarks on AT ................................................................. 54
3.5 Activity theory and design ..................................................................... 56
4 Evaluation of Groupware Systems ................................................................ 59
4.1 Overview of the four groupware systems ........................................... 59
Habanero .................................................................................................... 59
LOTUS LearningSpace ............................................................................. 61
5
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
BSCW Shared Workspace system ....................................................... 62
TeamWave Workplace ............................................................................ 63
4.2 Test Scenario Design ............................................................................... 64
4.3 Preliminary Analysis ................................................................................. 66
Lotus LearningSpace ............................................................................... 66
TeamWave Workplace ............................................................................ 66
Habanero .................................................................................................... 67
BSCW ........................................................................................................... 69
4.4 Discussion .................................................................................................. 71
The groupware systems .......................................................................... 71
Preliminary observations ........................................................................ 72
Preliminary suggests for what a groupware
system should contain ................................................................. 74
5 Teamwave workplace ....................................................................................... 77
5.1 The basis for design ................................................................................ 77
Creating a shared networked place .................................................... 80
Awareness ................................................................................................. 82
5.2 User interface considerations .............................................................. 84
5.3 Production, Communication and Distribution .................................... 85
Production/Knowledge construction .................................................. 86
Communication .......................................................................................... 87
Distribution/Division of Labour .............................................................. 88
5.4 Technical requirements .......................................................................... 89
5.5 Final remarks on TeamWave Workplace ............................................ 90
6 Ideels .................................................................................................................... 93
Norwegian Participation ......................................................................... 95
OPUSi ........................................................................................................... 96
6.1 Evaluation of intra-group collaboration .............................................. 98
The Research Study ................................................................................. 99
Data Collection ........................................................................................ 100
Data Analysis ........................................................................................... 102
Analysing the context using the activity system ............................ 106
Conclusions ............................................................................................. 109
6.2 Changes in Attitude towards Simulation-based
Distributed Learning .............................................................................. 112
Pre-questionnaire ................................................................................... 112
Knowledge of computers ...................................................................... 117
7 VisArt .................................................................................................................. 121
7.1 VisArt Design ........................................................................................... 121
The design process ............................................................................... 122
Modelling the VisArt Activity ............................................................... 124
6
Preface
7.2
7.3
7.4
7.5
7.6
7.7
7.8
7.9
7.10
7.11
7.12
Management and Coordination .......................................................... 126
Student Consent to Participate in Research ................................... 129
The Computer Supported Collaborative Learning Assignment ... 131
The Background Literature .................................................................. 131
The VisArt Activity .................................................................................. 132
The Individual Report ............................................................................ 135
Communication VisArt ............................................................................ 136
Assistance in VisArt ............................................................................... 136
VisArt in TeamWave Workplace ......................................................... 137
The Classroom ......................................................................................... 138
The Training Room .................................................................................. 139
Deployment of VisArt ............................................................................. 151
Participants .............................................................................................. 152
Preparation for VisArt ............................................................................ 153
Training ...................................................................................................... 153
The Design Actvitiy ................................................................................ 154
Reflections on the Design .................................................................... 158
Assistance and Help .............................................................................. 158
Administration of VisArt ......................................................................... 161
Evaluation of VisArt ................................................................................ 164
The learners’ experiences ................................................................... 165
Sharing of information ........................................................................... 165
Division of labor ...................................................................................... 166
Joint thinking ........................................................................................... 166
Awareness ............................................................................................... 166
A Usability evaluation of TeamWave Workplace ........................... 168
Research design ..................................................................................... 168
Analysis and Preliminary conclusions ............................................... 171
Effectiveness ........................................................................................... 171
Efficiency ................................................................................................. 173
PePedagogical decisions: .................................................................... 173
Satisfaction .............................................................................................. 177
Conclusions ............................................................................................. 182
The Role of Classical Jungian Personality Factors in
Computer Supported Collaborative Learning Environments ........ 183
Review of Previous Personality Research in the use of
predicting and understanding user attitudes and
behaviour ...................................................................................... 183
Previous work by the author ................................................................ 184
The Myers-Briggs Personality Test .................................................... 184
The Myers-Briggs personality Type Inventory
7
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
(MBTI) in Education ................................................................... 185
Method ...................................................................................................... 186
Security and ethical controls .............................................................. 186
Data entry and analysis ........................................................................ 187
Results ....................................................................................................... 188
7.13 Evaluating How Students Organise their Work ............................... 191
Evaluation approach ............................................................................. 192
The data collection ................................................................................ 192
The data .................................................................................................... 194
Preliminary data analysis ..................................................................... 195
Co-ordination ........................................................................................... 196
Communication mode ............................................................................ 198
Feedback .................................................................................................. 200
Summary and preliminary findings ..................................................... 201
7.14 Evaluating the Organisation of a Collaborative Telelearning
Scenario .................................................................................................... 202
Methodological aspects ...................................................................... 202
Data collection ........................................................................................ 205
Preliminary data analysis ..................................................................... 205
7.15 Supporting Collaborative Telelearning Research using
Server Logs .............................................................................................. 211
Method ...................................................................................................... 212
The TeamWave Workplace Server File ............................................. 212
The TeamWave Workplace Version Control .................................... 213
Two tools for analysing log-files ......................................................... 214
Conclusion ............................................................................................... 220
7.16 Coordination in collaborative telelearning ...................................... 221
Method ...................................................................................................... 222
Analysis ..................................................................................................... 223
Summary .................................................................................................... 224
7.17 Facilitating Help and Training in VisArt ............................................. 225
Data collection ........................................................................................ 225
Preliminary analysis ............................................................................... 226
8 Results and conclusions ............................................................................... 231
8.1 Collaboration Patterns ........................................................................... 231
Finding collaboration patterns ............................................................ 231
Four collaboration patterns ................................................................. 232
Pattern 1: Adaptation ............................................................................. 232
Pattern 2: Coordinated desynchronisation ...................................... 233
Pattern 3: Constructive commenting .................................................. 235
Pattern 4: Informal Language .............................................................. 236
8
Preface
Structure of collaboration patterns ................................................... 237
How we plan to use collaboration patterns in further work ........ 238
8.2 Studying students’ online learning .................................................... 240
Methodological issues .......................................................................... 241
Online and offline activities ................................................................. 243
9 Summary and Conclusions ............................................................................ 245
9.1 Summary .................................................................................................... 245
9.2 Conclusions ............................................................................................. 249
References ............................................................................................................. 253
Appendicies ........................................................................................................... 267
9
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
List of Figures
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
Figure 27
Figure 28
Figure 29
Figure 30
Figure 31
Figure 32
Figure 33
10
The DoCTA Project’s home page ..................................................... 22
Vygotsky’s mediating triangle (Vygotsky, 1978) ............................ 44
The basic mediational triangle (Cole & Engeström, 1993) ......... 47
Engeström’s (1987) model of the activity system ......................... 48
The hierarchical structure of activity (Leontev, 1978) ................ 51
The nested nature of activity dynamics (Engeström 1987) ....... 53
Integration of a design perspective with Activity Theory
(Mørch & Wasson, 1999) ................................................................... 57
The physical setting of the scenario .............................................. 65
The two white boards used by the students in Habanero ........ 68
Part of the class-diagram the students made in MS paint
while testing BSCW ............................................................................ 70
Barbara Wasson’s (empty) Room in TeamWave Workplace ..... 80
Room and permissions menus in TeamWave Workplace .......... 81
Classroom – An example of a TW room and its tools .................. 82
Screenshot from the IDEELS’ homepage ....................................... 94
Supporting tools in the scenario ..................................................... 96
Screenshot from OPUSi ...................................................................... 98
The two phases in the collaboration process............................ 103
The collaboration process as an activity system ...................... 107
The hierarchical structure of an activity ..................................... 109
Constraints on the VisArt design ................................................... 122
Top level of VisArt model ................................................................. 125
A second level of the model of VisArt .......................................... 126
Organisation of Assistance in VisArt ............................................ 137
The Classroom in VisArt ................................................................... 138
Training Room in VisArt .................................................................... 139
Welcome to the Training Room ...................................................... 140
Training Assignment 1 room in VisArt ........................................... 141
Training Assignment 1 ...................................................................... 142
An excerpt from the Narg Island Game ....................................... 143
Directions for a Narg Island Game with Distributed Teams .... 144
Training Assignment 4 ...................................................................... 146
VisArt’s Help Room in TeamWave .................................................. 147
Home Page of TeamWave Workplace Help Pages
(developed for VisArt) ...................................................................... 148
Preface
Figure 34
Figure 35
Figure 36
Figure 37
Figure 38
Figure 39
Figure 40
Figure 41
Figure 42
Figure 43
Figure 44
Figure 45
Figure 46
Figure 47
Figure 48
Figure 49
Figure 50
Figure 51
Figure 52
Figure 53
Figure 54
Figure 55
Figure 56
Figure 57
Figure 58
Figure 59
Figure 60
Figure 61
Figure 62
Figure 63
Figure 64
Figure 65
Figure 66
Figure 67
Figure 68
Help page for the Address Book tool ........................................... 149
Help page for the File Holder tool ................................................. 150
Help page for Navigating between rooms .................................. 151
Team 7’s working room ..................................................................... 154
Team 7’s room for learning about polar bears ............................ 155
Team 10’s group room ....................................................................... 156
Team10’s warehouse ........................................................................ 157
Designed learning room for Team10 ............................................. 157
Ease of getting help/information ................................................... 160
Administrator Interface for server information and sending
messages ............................................................................................ 162
Administrator interface for creation of accounts ..................... 162
Administrator interface for creation of groups .......................... 163
TeamWave Workplace tools grouped according to
functionality ....................................................................................... 170
An instructional room designed by Team 6 ................................. 172
Extracts from Team 6’s pedagogical decisions file ................... 173
Extracts about asynchronous communication in the
interviews ............................................................................................ 175
Extracts about synchronous communication in the
usability interviews ........................................................................... 176
Some example questions from the pre-questionnaire .............. 178
Perception of TW regarding usefulness, simplicity and
attractiveness. ................................................................................... 178
Difficulty of understanding functions in TW .............................. 179
Stability of TW server ....................................................................... 179
Missing functions in TW ................................................................. 180
Categories of negative and positive comments ........................ 181
Producing a TW learning room as an activity system .............. 195
VisArt instructors and facilitators as an activity system ......... 206
Extractions from the VisArt server.log file ................................... 213
An excerpt from SLA’s group log-file ............................................. 215
Graphs of Team Activity in VisArt .................................................. 216
One of many possible views of TRA .............................................. 218
Using TRA to record analysis of tool use .................................... 220
Experience with collaboration from prior education ................ 226
Experience with collaboration from work life ............................ 227
Usefulness of the training in TW, after completion of the
design task ......................................................................................... 227
Help page for the File Holder tool ................................................. 370
Help page for the ToDoList tool ..................................................... 370
11
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 69
Figure 70
Figure 71
Figure 72
Figure 73
12
Help page for the Page function ................................................... 371
Help page for how to Save Rooms ................................................ 371
Administrator interface for administering conferences .......... 372
Administrator interface for administering templates ................ 373
Administrator interface for administering repositories ............ 373
Preface
List of Tables
Table 1
Table 2
Table 3
Table 4
Table 5
Table 6
Table 7
Table 8
Table 9
Table 10
Table 11
Table
Table
Table
Table
Table
Table
12
13
14
15
16
17
Table 18
Table 19
Table 20
Table 21
Table 22
Table
Table
Table
Table
Table
23
24
25
26
28
Research Questions asked in the DoCTA studies ....................... 20
Project DoCTA Work Package Deliverables .................................. 24
Dependencies between Activities
(from Malone & Crowston, 1994) ...................................................... 35
Dependencies between Actors ....................................................... 37
TW tools facilitating production/knowledge construction
(Fjuk et al. 1999) ................................................................................... 87
How TW facilitates communication (Fjuk et al. 1999) ................. 88
How TW facilitates distribution/division of labour (Fjuk et al.
1999) ....................................................................................................... 89
Data gathering techniques ............................................................. 100
Distribution of background information by Site ......................... 113
Distribution of ‘computer at home’ and ‘
computer use’ by Site. ..................................................................... 114
Distribution of experience with teamwork, role playing
and negotiating games by Site. ..................................................... 114
Evaluation of the simulation by Site ............................................. 115
Evaluation of the simulation by Gender ....................................... 116
Cross-cultural experience by Gender........................................... 116
Experience with teamwork by Gender ......................................... 117
Overall evaluation by Gender ......................................................... 117
Overall evaluation of the simulation by
Knowledge of computers ................................................................ 118
A good learning experience (attitude change) * Gender ....... 118
The simulation game will be/was difficult
(attitude change) * Gender ............................................................ 119
A good learning experience * The simulation game
will be/was difficult ........................................................................... 119
Preparation from the Instructor’s Perspective
(as at 24.02.99) ................................................................................... 127
Preparation from Researcher’s Perspective
(as at 24.02.99) ................................................................................... 129
The 5 VisArt Activities ...................................................................... 133
Training Assignment 3 ...................................................................... 145
Data gathering techniques ............................................................. 194
Instructor and facilitator roles in VisArt ...................................... 203
Characteristics of Collaboration Patterns ................................... 238
13
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
List of Appendicies
Appendix
Appendix
Appendix
Appendix
Appendix
Appendix
Appendix
Appendix
Appendix
Appendix
DoCTA Publications ................................................................. 268
DoCTA Presentations .............................................................. 270
Assignment for test-scenario (English) .............................. 272
Interview Guide for Evaluation of Groupware Systems .. 275
The IDEELS Scenario .............................................................. 276
Assignment for designing an evaluation of IDEELS ........ 290
Interview Guide for IDEELS (English) .................................. 292
Pre-simulation survey for IDEELS ......................................... 294
Post-simulation survey for IDEELS ....................................... 299
Planning Tables from the Instructor’s View
(Phases II-IV) ............................................................................ 303
Appendix K: Planning Tables from the Researcher’s View
(Phases I-III) ............................................................................. 305
Appendix L:
Planning Tables from the Student’s View (Phases I-III) . 307
Appendix M: VisArt Consent Form ................................................................ 310
Appendix N: Semester Assignment ............................................................. 312
Appendix O: Directions for TeamWave Workplace Installation .......... 317
Appendix P: Designed Learning Rooms from VisArt ................................ 318
Appendix Q: VisArt Profile Questionnaire (Norsk) ................................... 323
Appendix R: VisArt Self-Evaluation Questionnaire .................................. 328
Appendix S: VisArt Pre-questionnaire ........................................................ 331
Appendix T:
VisArt Post-questionnaire (Norsk) ....................................... 340
Appendix U: Norweign Version of Training Assignments 1,3 & 4 ......... 354
Appendix V: The Narg Island Game ............................................................ 357
Appendix W: Help Page Screen Shots ......................................................... 370
Appendix Y:
Administration Interface in TeamWave Workplace ........ 372
Appendix Y:
Usability Study Interview Guide ........................................... 374
Appendix Z: Interview Guide for How Students Organise their Work 380
Appendix AA: Interview Guide for How Instructors and Facilitators
Organise their Work (English) ............................................... 384
14
A:
B:
C:
D:
E:
F:
G:
H:
I:
J:
Preface
Preface
Activities of project DoCTA: Design and use of Collaborative Telelearning
Artefacts (project leader Barbara Wasson) are aimed at the design
and use of technological artefacts to support collaborative
telelearning aimed at teacher training. The research is not limited
to only studying these artefacts per se, but includes social, cultural,
pedagogical and psychological aspects of the entire process in which
these artefacts are an integral part. This means that we both provide
and study virtual learning environments that are being deployed to
students organised in geographically distributed teams. The main
research focus is reflected in both the theoretical and methodological
approach chosen in the project. The theoretical, or conceptual
approach, is rooted in a sociocultural perspective that emphasises an
understanding of language, culture and other aspects of social setting. The methodology is influenced by ethnographic studies,
favouring naturalistic and qualitative research methods.
Three scenarios utilising the Internet were used to engage the students in
collaborative learning activities. Scenarios IDEELS and Demeter involved
European inter-cultural simulations where the goal was to design a textual
artefact (e.g., a treaty or policy statement). A third scenario, VisArt, was
designed and developed explicitly for use between the three educational
partners and had the goal of designing a visual artefact to be used in
teaching a subject of choice. These scenarios were studied from a number of perspectives including ethnographic flavoured studies focused on
understanding work organisation, usability studies of groupware systems,
evaluations of computer logs, and questionnaire studies.
This report documents the activities undertaken in project DoCTA. The
majority of the studies were undertaken as part of graduate students’
thesis work. Although all data has been collected, analysis of the data is
15
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
still ongoing, and the dissertations are in various stages of being written.
Thus, our final discussion in this report is preliminary and can only point
to the direction in which our integrative analysis is headed.
DoCTA has been administered and co-ordinated by the Department of
Information Science at the University of Bergen (Bergen). It has been a
collaboration with HiNT (Høgskolen i Nord-Trøndelag), HSH (Høgskolen
Stord/Haugesund) and Telenor FOU, Kjeller. The 19 researchers and
graduate students involved have various backgrounds including computer science, psychology, sociology and education. Project members
included:
University of Bergen
Nord-Trøndelag
College
College
Barbara Wasson Hege Higraff
Knut Ekker
Anders Mørch
Øyvind Meistad
Arnstein Eidsmo
Konrad Morgan
Trond Pedersen
Glenn Munkvold
Mike Spector
Kurt Rysjedal
Eskil Andreassen Helge Underhaug
Rune Baggetun
Jo Wake
Frode Guribye
Stord/
Haugesund
Lars Vavik
Amund Junge
Telenor FOU,
Kjeller
Annita Fjuk
As project leader I take the liberty to thank all project members for
efforts both during the numerous project activities and, especially, during these last months as we have prepared for the delivery of this final
report. In particular I thank my D.Polit student Frode Guribye for all his
extra efforts while I am on maternity leave. In addition, we all thank Dr.
Ileana de la Teja of LICEF, Télé-université, Montréal who served as a
consultant on the design of VisArt. Finally, I would like to thank KUF
(ITU programme) for funding DoCTA and supporting 6 of our graduate
students with stipends.
Barbara Wasson, January, 2000
16
Chapter 1 Introduction
1 Introduction
Project DoCTA (Design and use of Collaborative Telelearning Artefacts)
aims to bring a theoretical perspective to the design of ICT technologies
that supports the sociocultural aspects of human interaction and to
evaluate its use. The main, long term, research objectives are to:
• take a sociocultural perspective on learning activity focussing on the
interpersonal social interaction in a collaborative learning setting
(collaboration = coordination + communication)
• contribute to collaborative telelearning knowledge about the
pedagogical design of learning scenarios, the technological design
of the learning environment to support these learning scenarios, and
the organisational design for management of such learning
environments, including a reflection on teacher and learner roles
• study and evaluate the social and cultural aspects of distributed
collaborative telelearning environments
Through these objectives we aim to improve our understanding of the
pedagogy and technology of networked learners, and increase our
understanding of learner activity. This will lead to better design, management and affordances (Gibson, 1986; Laurillard, 1987; Norman, 1990)
of on-line learning spaces.
In the first phase of project DoCTA (June 1998 - December 1999) we
focus on the design and use of technological artefacts to support
collaborative telelearning aimed at teacher training (Wasson & Mørch,
1999). The research is not limited to only studying these artefacts per
se, but includes social, cultural, pedagogical and psychological aspects
of the entire process in which these artefacts are an integral part. This
means that we both provided and studied virtual learning environments
that were deployed to students organised in geographically distributed
teams. The main research focus is reflected in both the theoretical and
17
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
methodological approach chosen in the project (Wasson & Guribye, 1999;
Mørch & Wasson, 1999). The theoretical, or conceptual approach, is
rooted in a sociocultural perspective and the methodology was influenced
by ethnographic studies, favouring naturalistic and qualitative research
methods. These ethnographic flavoured studies were augmented with
more traditional computer science oriented usability studies, evaluations
of computer logs, and questionnaire studies.
Various scenarios utilising the Internet were used to engage the students
in collaborative learning activities (Wasson & Mørch, 1999). Through
participation, the teachers gained experience with not only collaborative
learning, but with collaborative telelearning through the collaborative
design of a textual or visual artefact. Scenarios IDEELS and Demeter,
involved European inter-cultural simulations where the goal was to design a textual artefact (such as a treaty or policy statement). Scenario
VisArt (Wasson, 1999b) had the goal of designing a visual artefact to be
used in teaching a subject of choice. The IDEELS and Demeter scenarios
resulted in 1 Masters1 dissertation each, Guribye (1999) and Junge, (1999),
respectively. VisArt will result in 9 Masters dissertations ¾ these are
currently being written. In addition, two researcher studies, one of IDEELS
(Ekker, 1999) and one of VisArt (Morgan & Morgan, 1999) have been
completed. Although we have begun to integrate the results from the
individual studies in order to identify collaboration patterns, there is much
work to be done.
The remainder of this introduction focuses on DoCTA’s research questions,
enumerates our evaluation studies, points out highlights of the first phase
and describes the organisation of the report.
1.1
Research Questions
Project DoCTA is first and foremost an exploratory study that focuses on
understanding interaction in a set of collaborative telelearning scenarios.
The community of study includes teachers, learners and facilitators
participating in the various collaborative telelearning scenarios. The study
provides us with insight into the processes of collaboration enabling us to
1
18
A Masters dissertation is the written document resultant from a Norwegian
hovedfagsoppgaver.
Chapter 1 Introduction
identify collaboration patterns and further our understanding of how
instructors, students and other learning facilitators organise their learning
and work given the kinds of artefacts they have access to, the kind of
learning activity they are presented with, and the kinds of artefacts they
will design.
While a plethora of research questions could be asked, we restricted this
study to two main questions. Given that within our collaborative
telelearning scenarios, the contexts vary with respect to the actor
characteristics, the type of learning activity, the kinds of artefacts they
have access to, and the kind of artefact to be designed, we ask:
• How do the students, teachers and facilitators organise their work?
• How do patterns of collaboration vary?
In order to answer these two primary questions, a number of evaluation
studies are being carried out on several levels and from several
perspectives. In these evaluations, more specific questions have been
asked. Eleven of these studies are part of Masters’ projects and cover:
evaluations of suitability of groupware systems for collaborative
telelearning; usability studies of TeamWave Workplace (TW); looking at
the efficiency of TW from a qualitative perspective using the data logs
generated by TW; performing a formative evaluation of how to support
collaborative design activities; seeing how TW supports coordination; how
to design training and assistance in a collaborative telelearning setting;
and, several activity theory studies of how students, instructors and
facilitators organise their work. In addition, as mentioned above, two
researcher studies, one of attitude changes over time and one on
personality factors and choice of collaboration artefacts have been
completed. Table 1 summarises the research questions of the various
studies.
Numerous data and data collection techniques have been used including
questionnaires, semi-structured interviews, automatically generated log
files, email sent among team members, instructors and facilitators, chat
files saved by the teams, documents and artefacts produced by the teams,
participant observations, video taping of activities, self-evaluations and
the participants own interpretation of the experience.
19
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 1 Research Questions asked in the DoCTA studies
Domain/theoretical framework Research Question
Suitability of Groupware Systems for Collaborative Telelearning
Evaluation of the interface and Can students working in collaboration solve a
the relative usefulness
collaborative design task with Habanero,
TeamWave Workplace, BSCW and Lotus
LearningSpace?
Usability of TeamWave Workplace
Usability – effectiveness
Can students working in collaboration reach their
team goals with TW?
Usability – efficiency
To what extent do the tools enable students to
meet their task?
Usability - satisfaction
Are students satisfied of TW tools?
Efficiency from a qualitative perspective
Comparison of anticipated use
and actual use of the tools
What are the differences between the anticipated
use of the tools and the actual use of the tools?
Which are the differences of tool use when alone
and when others in the room?
How are the rooms used?
Research methodology
What implications do the use of the tools have for
electronic data-collection?
Formative evaluation
Tailoring for instructional design How to support pedagogical room design in TW?
User centred design
How do students use TW tools to design a room for
teaching?
What is to be improved?
Attitude changes
What changes in attitudes towards simulationbased learning situations occur?
Personality factors
Are there any personality factors involved in the
use of tools in telelearning situations?
Coordination theory
Supporting coordination
How to support coordination in a collaborative
telelearning environment?
User centred design
What kind of coordination do students do?
20
Chapter 1 Introduction
Activity theory
Organisation of work
How do the students organise their work?
– student perspective
Organisation of work
– Instructor‘s perspective
How instructors and facilitators organise their
work
Design of training and assistance
Training / Assistance
effectiveness
How effective is the training designed for
collaborating and learning to use TW?
How to improve training and assistance?
1.2
Highlights
During the past one and a half years, there have been many activities
around project DoCTA. This section discusses some of the highlights.
DoCTA Workshops: Project DoCTA began with a kick-off workshop
in Bergen in September 1998. We arranged and hosted an Activity
Theory workshop in October 1998 with Dr. Victor Kaptelinin as the
invited speaker (2 graduate students from the ITU Project Ny Teknologi
og NyePraksisformer also attended this workshop). In November 1998
we had an internal Bergen workshop on the Design of VisArt. Finally, a
VisArt Debriefing workshop was held in the late spring of 1999 after
the VisArt scenario was completed. This workshop served as a means of
debriefing ourselves about the scenario.
Publications: Project DoCTA has resulted in 9 publications. One
additional paper is under review and another was invited to appear in
the IFETS (The Journal of the International Forum of Educational Technology & Society) Special Issue on On-line Collaborative Learning
Environments. Appendix A lists these publications. Guribye & Wasson
(1999) was awarded an Outstanding Paper award at Ed-Media’99 and
Meistad (1999) was one of three winners of the ITU Essay Contest.
21
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Presentations: Seventeen presentations of DoCTA have been presented.
These presentations, listed in Appendix B, have presented various aspects
of the DoCTA project.
Conferences: DoCTA researchers attended 10 different conferences,
workshops or graduate summer schools. The project was presented at
many of these. Several of the graduate students were able to attend
conference tutorials that provided both relevant information and motivation
for their Masters thesis work.
Masters’ dissertations: The IDEELS and Demeter scenarios resulted
in 1 Masters dissertation each, Guribye (1999) and Junge, (1999),
respectively. The VisArt scenario will result in 9 Masters dissertations Andreassen (in preparation), Baggetun (in preparation), Higraff (in preparat-
Figure 1
22
The DoCTA Project’s home page
Chapter 1 Introduction
ion), Meistad (in preparation), Pedersen (in preparation), Rysjedal (in
preparation), Underhaug (in preparation), and Wake (in preparation).
Hovedfagforum: Six of DoCTA’s graduate students received stipends
from ITU’s hovedfagforum (graduate student forum). This enabled them
to attend conferences and tutorials.
Web page: Project DoCTA strives to maintain a project web page. Figure
1 presents the project’s home page. The pages include information about:
the four scenarios; the project partners and researchers, publications
and presentations, and a set of resources including related projects,
conferences (with reviews), relevant literature (with a review), and relevant web pages. We are currently adding an overview of the VisArt scenario that will provide a demo tour through the scenario and include a
presentation of some of the visual artefacts created by the students
participating in the scenario.
1.3
Organisation of the Report
This report is rather substantial and serves as the final report for the first
phase of project DoCTA. As our integrative analysis is very preliminary
at this point2 , subsequent results will be published as Masters dissertations,
or conference or journal papers.
This report is divided into three parts. Part I expands the theoretical
foundations and conceptual framework described in the DoCTA project
application. Part II presents the design, deployment and evaluation of
each two of the scenarios, IDEELS and VisArt. Part III presents some of
the tentative results on collaboration patterns.
In the project application we identified a number of work packages and
deliverables. Table 2 lists these work package deliverables and identifies
where in the report they can be found3 .
2
3
The project leader has been on maternity leave since July.
These are often only summaries or examples of what will be reported in a
Masters dissertation.
23
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 2 Project DoCTA Work Package Deliverables
WP#
Deliverable
Source
Where in Report
2.1
Expanded theoretical
framework
Hovedfag
dissertations,
published papers
Part I, section 2
2.2
Review of possible tools for
use in WP 7
Hege’s dissertation,
Barbara’s review
Part I, sections 4 & 5
3.1
A set of quantitative and
qualitative tools (e.g.,
questionnaires, interview
guides)
IDEELS questionnaires,
VisArt questionnaires,
Interview schemes
Appendices
D, G-I, M-O, Q-Z, AA
3.2
A description of our
qualitative method (including
data collection issues, data
analysis issues, AT, etc.)
Frode’s dissertation,
published papers
Part I, section 3
Part III, section 7.10 – 7.17
3.4
A scenario 1 evaluation
report (IDEELS)
Frode’s dissertation,
Knut’s study
Part II, section 6.1
Part II, section 6.2
3.6
A scenario 3 evaluation
report (VisArt )
Hovedfag dissertations,
published papers
Part II, section 7
3.7
Final integrating evaluation
report
Group meetings,
evaluation studies,
collected data
Part III, section 8.1
4.1
Documented PedInfo learning EdMedia paper
scenario (this scenario
changed from the original
application)
See Wasson (1999b)
4.3
Documented experience with Trond’s experience on
the TeamWave Workplace
administration of TW
software
Part II, section 7.8
5.1
Design details of the IDEELS
scenario and reflections on
the deployment
Frode’s dissertation,
Part II, section 6
Bergen student’s reflection,
HiNT reflections (Glenn,
Knut & Arnstein), published
papers, Knut’s questionnaire
6.1
Design details of the scenario
and reflections on the
deployment
Amund’s thesis: Junge (1999)
7.1
Design details of the
VisArt scenario
Scenario tables,
VisArt assignment
published papers,
hovedfag dissertations
24
—
Part II, section 7.1-7.5
Appendicies J-O
Chapter 1 Introduction
7.2
FAQ-list for commercial tools
Kurt’s help pages, help
room, tips
http://www.ifi.uib.no/docta/
VisArt/help
Part II, section 7.5
http:/www.ifi.uib.no/docta/
teamwave/install.html,
7.3
Reflections on deployment
Trond’s administration
Part II, section 7.6
experience,
Helge/Barbara/Trond on
help/assistance
7.4
Web Archive
All contribute with screen Under development, will
shots of your teams’ rooms be accessible from the
DoCTA web page
http://www.ifi.uib.no/docta
8.1
DoCTA web site
All
8.3
International publications
List on web & in Appendix Appendix A
http://www.ifi.uib.no/docta
Part I: Theoretical Foundations and Conceptual
Framework
Part I expands the theoretical foundations and conceptual framework
presented in the project application, presents a review of four groupware
systems and concludes with a comprehensive description of TeamWave
Workplace. Section 2, theoretical foundations, covers computer supported
collaborative learning, genuine interdependence, collaborative
telelearning, coordination theory, awareness, and distributed collaborative
learning communities. Section 3 presents the conceptual framework that
has influenced the evaluations of the scenarios. In particular, Vygotsky
and cultural-historical psychology are reviewed and Activity Theory is
covered in detail. The section concludes with some general remarks on
AT and on it’s implications for design. Section 4 presents an analysis of
four groupware systems, Habanerio, Lotus LearningSpace, BSCW Shared
Workspace System, and TeamWave Workplace. Finally, a thorough
review of TeamWave Workplace in presented in section 5.
Part II:The Scenarios
Part II has two sections, one for each scenario. Section 6 reports on the
IDEELS scenario and section 7 on VisArt. Each section describes its
scenario’s design, discusses issues around the deployment of the scenario, and then presents evaluation studies of the scenario.
25
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Part III:
Results and Conclusions
Part III has 2 sections. Section 8 discusses the results of the project to
date. Section 8.1 reports the tentative results of our integrative analysis
of collaboration patterns. As indicated in the preface, however, the project
provided the opportunity for 11 Masters theses and these are in various
stages of completion. Therefore, the integrative analysis of collaboration
patterns presented in this report is preliminary. Section 8.2 presents a
discussion of methodological issues around virtual ethnography. Section
9 summaries the report and gives some conclusions.
26
Chapter 2 Theoetical foundations
Part I
Theoretical
Foundations,
Conceptual
Framework, and
Groupware Systems
27
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
28
Chapter 2 Theoetical foundations
2 Theoretical foundations
BARBARA WASSON
There are a number of research areas that have influenced project DoCTA
(see Wasson & Mørch, 1999; Wasson, 1999 for details). The two most
significant are the conceptual framework offered by sociocultural perspectives
and the emerging area of computer-supported collaborative learning, in
particular Salomon’s (1992) work on genuine interdependence. Other
perspectives that provide inspiration and guidance for this research come
from: computer supported collaborative work (CSCW), in particular Gutwin
et al.’s (1995) ideas on awareness; coordination science (Malone & Crowston,
1994); sociocultural perspectives (Wertsch, del Río & Alvarez, 1995) on
learning and thinking; and, the emerging notion of distributed learning
communities. The conceptual framework adopted in the project is described
in section 3. This section provides a brief description of the remaining areas.
2.1
Computer supported collaborative learning
Computer supported collaborative learning (CSCL) is an emerging paradigm
(Koschmann, 1996) for research in educational technology that focuses on
the use of information and communications technology (ICT) as a mediational
tool within collaborative methods (e.g., peer learning and tutoring, reciprocal
teaching, project- or problem-based learning, simulations, games) of learning.
It is an approach to ICT in education that emphasises an understanding of
language, culture and other aspects of the social setting (Scott, Cole & Engel, 1992). Its intellectual heritage can be found in social constructivism
(Doise, 1990), the Soviet cultural-historical psychology (e.g., Vygotsky (1978),
Leontiev (1978), Davydov (1988)) and situated cognition (Suchman, 1987;
Lave, 1988).
CSCL research involves naturalistic observations being used in an exploratory
29
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
fashion to permit a more complete understanding of this instructional mode.
The emphasis is on the process and not so much focused on outcome.
Evaluations often result in descriptive studies which focus on artefacts that
support or are produced by teams of learners and usually contain participant
accounts of their own work. The influence of CSCL research on DoCTA is
evident in the type of research questions we ask and in the choice of
conceptual framework for organising our evaluations.
Project DoCTA subscribes to the collaborative learning paradigm.
Collaborative learning can be seen as a collection of perspectives based
on principles of interpersonal interaction that emphasises an understanding of language, culture and other aspects of the social setting (Scott,
Cole & Engel, 1992). Traditionally methods such as peer learning and
tutoring, reciprocal tutoring, project- or problem-based learning,
simulations and games have been used to engage learners in collaboration.
Collaborative learning also refers to notions of “socially shared cognition”
(Resnick, Levine & Teasley, 1991), of “distributed cognition” (Salomon,
1993) and of “jointly accomplished performance” (Pea, 1993), which
emphasise that cognitive development occurs through interactions
between students, between students and teachers, as well as between
students and rich or knowledgeable environments.
ICT support for collaborative learning can take various forms. Fjuk (1998)
identifies three perspectives of collaboration that place emphasis on different goals including joint construction, joint negotiation and using
teachers and peers as resources. Andriessen & Sandberg (1999) list four
different uses of ICT to support collaborative learning including computer-based collaborative tasks, co-operative tools, computer mediated
communication, and intelligent co-operative systems. The Center for
Innovative Lear ning Technologies (http://cilt.or g/html/
communitytools.html) at the Stanford Research Institute (SRI) takes a
tools oriented approach and identifies three classes of interactive digital
media for collaboration including tools for collaborative representation,
tools for social and knowledge networking, and tools for cognitive
scaffolding. This can be summarised in the following perspectives on the
use of ICT to support collaborative learning:
1. Collaborative Tasks. In this perspective, the user is provided with an
environment that supports student-student collaborative problem
solving. Shared problem representation is an important aspect,
30
Chapter 2 Theoetical foundations
through, for example, visualisation and modelling. For one example
see the work on joint construction of knowledge (e.g., joint problem-solving by mutual refinement). For example, see the work on
CoVis (Pea, R., Edelson, D., & Gomez, L., 1994). On the web at
http://www.covis.nwu.edu/.
2. Co-operative or Collaborative Tools. In this perspective, a computer
serves to reduce the cognitive load or lower-order tasks so the student can focus on higher-order activities. Such tools provide cognitive
scaffolding. The computer provides cognitive support to structure
learning activity. For an example see the work on CSILE (Scardamalia
et al., 1989) where scaffolding in the form of sentence starters (e.g.,
I think., I know, I wonder, etc.) is provided (http://www.ed.gov/pubs/
EdReformStudies/EdTech/csile.html) Another example is the work
on Belvedere (Suthers & Weiner 1995) where students engaging in
joint negotiation of alternatives through argumentation, debate and
other means are provided with interface support for structuring their
arguments. On the web at http://advlearn.lrdc.pitt.edu/belvedere/
index.html
3. Computer-mediated Communication. In this perspective,
collaboration occurs over electronic networks. Students rely on peers,
teachers, domain experts, etc. as a resource to support their own
learning and for external feedback. For example Computer Mediated
Communication (CMC) systems or groupware systems (e.g., Teamwave
Workplace, http://www.teamwave.com) provide a forum where
distributed students can participate in collaborative learning activity.
4. Intelligent Co-operative Systems. In this perspective the computer
serves as an intelligent co-operative partner, a co-learner (Dillenbourg
& Self, 1992), or a learning companion (Chan & Baskin, 1990).
As designers of learning tasks and the technological support to go with
them, we must keep in mind that our foremost role is to provide a support environment that makes coordination, communication and
collaboration as transparent as possible (Bourdeau & Wasson, 1997).
This will enable students to create their own learning community.
31
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
2.2
Genuine interdependence
Salomon’s work on CSCL (Salomon, 1992, 1993) provides the most
complete approach to the study of CSCL in that it is built upon learning
theories, relies on observations, raises strong design issues and gives
methodological tools for educational research. Salomon (1992)
distinguishes between the effects with a tool and/or collaborating peers
and the effects of these. The first, the effects with, are the changes or
results afforded during the use of a tool or in collaboration with a partner
(i.e., the group can achieve more together than individually). The second, the effect of, is the aim Salomon advocates for education and is the
lasting changes, or cognitive residue, that are a consequence of an
intellectual partnership. To achieve long-term cognitive residue requires
mindful engagement on the part of the learner. Helping the learner
reach this level requires that “the whole learning environment, not
just the computer program or tool, be designed as a well orchestrated
whole. This includes curriculum, teachers’ behaviors, collaborative
tasks, mode of peer collaboration and interaction, tasks, learning
goals, and the like (Salomon, 1992, pg. 64)”.
Salomon’s research focus is on mediation in CSCL, which is a key issue in
collaborative telelearning. Salomon (1992, 1993) states that CSCL’s success
depends on mindful involvement and personal responsibility on the part of
the individual learner, and interdependencies between learners. Of primary
importance is interdependence. Furthermore, Salomon argues that effective
collaborative learning requires much and well orchestrated interdependence.
To avoid group phenomena such as the “free rider” (Kerr & Brunn, 1983),
the “sucker effect” (Kerr, 1983), the “status sensitivity” effect (Dembo &
McAuliffe, 1987) or the “ganging up on the task” phenomenon (Salomon &
Globerson, 1989), the interdependence must be genuine. Genuine
interdependence is characterised by Salomon (1992) as:
• the necessity to share information, meanings, conceptions and
conclusions
• a division of labour where roles of team members complement one
another in a joint endeavour and the end product requires this pooling of different roles, and
• the need for joint thinking in explicit terms that can be examined,
changed, and elaborated upon by peers.
In project DoCTA, Salomon’s ideas influence the design of the
collaborative tasks given to the students — effort is placed in designing
32
Chapter 2 Theoetical foundations
tasks that create genuine interdependence between the students. This is
illustrated in the design of the VisArt scenario (see Part II).
2.3
Collaborative telelearning
The term telelearning is used to designate new forms of distance or of
computer mediated learning, where the distance is not only distance in
space or time as in traditional distance learning, but the mediation of
learning activities served by media such as multimedia shared workspaces,
communication, or servers (Bourdeau & Wasson, 1997). Distance learning
evolved from a need to ensure equal access to education for all students
even when there are obstacles for them to access school or campus
(Bourdeau & Bates, 1997). The main features of distance learning are
the time and space co-ordinates, where students and professors do not
all meet at the same place at the same time. Individual learning, individual tutoring and asynchronous communication are typical features of a
distance learning situation, requiring extensive macro- and microinstructional design, and a strong student support system. These features,
however, do not dominate in telelearning. Many variations of telelearning
can be found in terms of presence, telepresence, meeting in virtual spaces,
interactivity with rich multimedia environments and extensive human
interactions in a virtual world with no limits except access and language.
Collaborative telelearning emphasises the collaborative interaction
between students in a virtual world. By following Salomon’s
recommendation for genuine interdependence, collaborative telelearning
tasks need to create interdependencies between team/group members.
In order to mange the interdependencies, team and group members will
need to coordinate their efforts. Providing support for this coordination
is one challenge for telelearning environments.
2.4
Coordination theory
(from Wasson (1997), Wasson (1998) and Wasson & Bourdeau (1997))
Schmidt (1997) argues that the investigation of artefacts for coordinative
purposes is a crucial and fertile area for CSCW (computer supported
coopeartive work) research. For CSCW researchers, cooperation is
33
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
defined as communication plus coordination (Olson et al., 1993). For
collaborative learning based on a definition of genuine interdependence,
both are certainly essential. Add collaborative learning to a telelearning
environment and this requirement for coordination is magnified.
Coordination entails the meshing, allocating, relating, and scheduling of
activities, actors and resources with respect to each other (Strauss, 1985;
Schmidt, 1994). Furthermore, the coordination of group activities requires
awareness information (Dourish & Bellotti, 1992). Awareness is “an
understanding of the activities of others, which provides a context
for your own activity (Dourish & Bellotti, 1992, p. 1)”.
Schmidt & Simone (1996) identify the understanding of how computer
systems can aid in reducing the complexity of coordinating cooperative
activities as a major research issue in CSCW. They argue that this issue
was identified and defined early in CSCW history by Holt (1985). He
wrote “Whatever has to do with task inter-dependence — coordination
— is left to the users to manage as best they can, by means of shared
databases, telephone calls, electronic mail, replies to which multiple
users have access, or whatever ad hoc means will serve (Holt, 1985)”.
Carsten & Sørensen (1996) point out that although many have explicitly
addressed and modelled coordination work (Holt, 1988; Malone &
Crowston, 1990, 1994; Kaplan et al, 1992; Fitzpatrick et al., 1995),
these studies have not been based on empirical studies of artefacts
introduced in order to cope with the complexity of the coordination work
the actors must handle. In their own field studies (Carsten & Sørensen,
1996) they found that actors themselves chose to increase the formalisation of work processes, despite the problems of increased rigidness
and possible loss of control of the work. Mechanisms were invented and
adopted to support coordination to deal with complexity.
Malone & Crowston (1994) describe coordination theory as an emerging
research area focused on the interdisciplinary study of how coordination
can occur in diverse kinds of systems. Coordination theory provides a
means for specifying (inter)dependencies between, and among, actors,
goals, activities, and resources by identifying a dependency type (e.g.,
shared resource) and a coordination process (e.g., group decision-making) for managing the dependency. In their work, coordination is defined
as managing dependencies between activities (Malone & Crowston,
1994). In particular, actors face coordination problems because of
dependencies between their activities, and as such, Malone and Crowston
have focused on dependence between activities.
34
Chapter 2 Theoetical foundations
Drawing on ideas about activity coordination in complex systems from
disciplines as varied as computer science, linguistics, psychology,
economics, operations research and organisation theory, they present a
first version of an analysis that characterises the basic processes involved
in coordination. Table 3 gives a taxonomy of dependencies between activities and possible coordination processes for managing them.
Table 3 Dependencies between Activities (from Malone & Crowston, 1994)
Dependency
Examples of coordination processes for managing
dependency
Shared resources
“First come/first serve”, priority order, budgets,
managerial decision, market-like bidding
Task assignments
(same as for “shared resources”)
Producer/Consumer relationships
Prerequisite constraints
Notification, sequencing, tracing
Transfer
Inventory management (e.g., “Just In Time”,
“Economic Order Quality”)
Usability
Standardisation, ask users, participatory design
Design for manufacturability
Concurrent engineering
Simultaneity constraints
Scheduling, synchronisation
Task / Subtask
Goal selection, task decomposition
For example, two activities are dependent if they share a resource. Resources
include actors, tools or information. A first come/first serve coordination
process would allow the first to ask for a resource (be it, for example, an
actor or a computer processor) to receive the resource. Another possibility
is that a managerial decision decides who gets the resource, or a marketlike bidding process takes place. Task assignment is a special case where
an activity needs to be assigned to an actor and the same coordination
processes that applied for a shared resource apply here. Another example
is that two activities are dependent if there is a prerequisite constraint
between them. In this case some kind of explicit sequencing and tracking
processes are needed. The usability dependence implies that the product
of one activity must be useable by the consuming activity and this can be
ensured by a standardisation process.
A simultaneity constraint dependency exists if two activities need to
occur at the same time. For example a meeting requires scheduling.
35
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The task/subtask dependency often surfaces when a goal is decomposed
into subgoals or activities. Coordination process goal selection is choosing
a goal while goal decomposition is the process of choosing activities.
Such a taxonomy of dependencies and related coordination processes
serves several purposes. First, the taxonomy serves as a way to organise
coordination mechanisms. This will enable others to determine if identified
dependencies or coordination mechanisms have already been recognised,
and if not, to add the new dependency type or new coordination
mechanism. Second, if you can identify a dependency, then you can
consult the table to find out what coordination process is needed to manage that dependency. The coordination process itself can be broken down
into a number of activities as is shown in table 3 where the generic
resource assignment coordination mechanism is represented as a four
step process comprising: identify necessary resource, identify available
resources, choosing resources and marking resources in use (i.e., assign
resource).
Although Malone & Crowston (1990, 1994) include actors in their theory,
the major emphasis is on the role of actors as a resource (an actor is
assigned to an activity). The coordination processes they identify deal
with managing the dependencies where it is a goal to minimise the number of dependencies in a complex system. In collaborative telelearning,
on the other hand, interdependence between learners is desirable4 , and
we design tasks that will create actor interdependence.
Wasson & Bourdeau (1997) report that viewing collaborative telelearning
from a coordination theory perspective offers a means of understanding
the inter-relationships between actors and entities and how these
relationships can and should be supported. Adopting Salomon’s ideas
about genuine interdependence and a coordination science approach,
they modelled (inter)dependencies between actors in collaborative
telelearning scenarios and they have extended the definition of
coordination to be managing dependencies between activities (Malone
& Crowston, 1994) and supporting (inter)dependencies between actors.
Wasson (1997, 1998) proposes a set of actor (inter)dependencies and
related coordination processes for collaborative telelearning.
4
36
Remember Salomon’s arguments that genuine interdependence is need for
successful CSCL
Chapter 2 Theoetical foundations
Table 4 presents an initial taxonomy of (inter)dependencies between
actors and gives examples of coordination processes for supporting the
(inter)dependencies. Both collaborative interdependence and competitive
interdependence can manifest themselves through either a shared goal
or a shared activity. Where they differ the most is in the coordination
processes required for managing them. Note that several of the
coordination processes are found in Salomon’s definition of genuine
interdependence
Table 4 Dependencies between Actors
Dependency
Examples of coordination processes for managing
dependencies
Collaborative Interdependence
shared goal
team building
shared activity
team assignment; team building; division of labour;
sharing of information, conceptions and conclusions;
joint thinking; group decision making; planning; task
assignment; scheduling; synchronisation; communication
Competitive Interdependence
shared activity
scheduling; communication; monitoring;
competition assignment
shared resource
same as in Table 3
The importance of this work for DoCTA lies in making sure the
technological environment within which the students carry out their tasks
provides mechanisms that make coordination as effortless as possible.
Otherwise coordination issues can become a bottleneck that hinders
students from carrying out their individual and collaborative work. Wasson
and Bourdeau’s extended definition is particularly important in the view
of collaborative telelearning as supporting a distributed collaborative
learning community where the aim is to support the learners in creating
that community.
2.5
Awareness
In 1992, CSCW researchers Dourish and Bellotti introduced the concept
of awareness and defined it as an “understanding of the activities of the
others, which provides context for your own activity. (Dourish & Bellotti,
37
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
1992, p. 107)”. Since that time it has been a concept that has received
a lot of attention (for example see Gutwin et. al., 1995; Bellotti and Bly,
1996; Palfreyman and Rodden, 1996; Roseman and Greenberg, 1996).
Gutwin et al. (1995) present a framework of awareness for collaborative
learning which comprises four types of awareness: social awareness, task
awareness, concept awareness and workspace awareness. Social awareness
refers to awareness of the social connections within the group or team
(e.g., What role will I take in the team, what roles will the other members
take). Task awareness concerns awareness about how to complete the
common task (e.g., What do we know about the task, how much time do
we have, what steps must we take to complete the task). With concept
awareness the individual student is aware of how a particular activity or
piece of knowledge fits into his or her own existing knowledge. Workspace
awareness is an up-to-the-minute knowledge of the other students’
interactions with the workspace (e.g., what are they doing, where are
they, what has been done so far, what is left to do). They suggest that social
awareness, being inter-personal, is best supported implicitly by providing
communication opportunities for team members to negotiate their roles.
Task and concept awareness are often supported by providing explicit
scaffolds to assist with organisation and helping stay on task. Workplace
awareness is maintained by providing “tracking information such as other
learners’ location in the shared workspace, their actions, the interaction
history and their intentions (Gutwin et al., 1995, p. 147).
In DoCTA, Gutwin et al.’s framework has been used by the students
participating in VisArt to evaluate whether TeamWave Workplace supports his categories of awareness.
2.6
Distributed collaborative learning communities
The final area providing inspiration to project DoCTA is summarised as
follows
“…a distributed collaborative learning community is a ‘place’ that is
created by the individual students through their individual and collective
actions, …The designers’ role is to support the students’ work of creating
that community, and in such a way that the computer systems become
integrated parts of the students’ activity (Fjuk, 1998, p. 70)”.
38
Chapter 2 Theoetical foundations
Furthermore, Fjuk (1995) concludes that collaborative telelearning
applications need to have both a mediating role between the individual
learner and the peer-students and between the individual learner and
her learning tasks. Thus, collaborative telelearning can be understood as
a medium for inter-human interactions and articulation of individual work.
As designers of DoCTA learning tasks and the technological support
environments, we must keep in mind that our role is to provide a supporting environment that makes coordination, communication and
collaboration as transparent as possible (Bourdeau & Wasson, 1997)
enabling students to create their own learning community. The environment must also support both individual and collaborative work.
39
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
40
Chapter 3 Conceptual framework for evaluations
3 Conceptual framework for
evaluations
FRODE GURIBYE
The underlying conceptual framework adopted in DoCTA is taken from
three different, although closely interrelated approaches, namely: activity
theory (Leontev, 1978, Engeström 1987), distributed cognition (Hutchins,
1995), and situated action (Suchman, 1987, Lave, 1988, Mantovani,
1996). One of the goals of Guribye’s (1999; Wasson & Guribye, 1999)
research5 is to argue that, together, these approaches make up a rich
framework for describing, evaluating and analysing collaborative
telelearning scenarios. All three approaches underscore the need to look
at real activities in real situations (Nardi, 1996), and always, in some
way, include the context in studies of human activity.
The rationale for combining these three approaches as the conceptual
foundation of this study, is that they all fall under what is called a sociocultural
perspective (Wertsch, del Río & Alvarez, 1995), that highlights learning
and thinking as phenomenon that cannot be studied in isolation. Rather,
they are complex processes situated or distribute in an environment — it
is impossible to separate them from the context in which they occur. The
different approaches each emphasise slightly different elements of the
framework that is important to be aware of in these kinds of studies.
Situated action emphasises the emergent, contingent nature of human
activity, the way activity grows directly out of the particularities of a given
situation (Nardi, 1996). Distributed cognition on the other hand, asserts
as a unit of analysis a cognitive system composed of individuals and the
artefacts they use (Hutchins, 1991, Nardi, 1996). This approach
5
This work began with his Masters research (Guribye 1999) and is being
continued in his Dr.Polit.
41
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
underscores the distributed nature of cognitive processes, and the role
that different artefacts play in these processes. Activity theory also
emphasises the mediating role of artefacts, but stresses that these artefacts carry with them a particular culture and history, thus, focuses on
the institutional and cultural elements involved in the learning activity
(Kuutti, 1996). In this report we confine our focus to activity theory.
Activity theory (AT) builds on a solid intellectual heritage (see e.g., Vygotsky,
1978; Zinchenko, 1995), provides extensive theoretical and conceptual
resources, places emphasis on the mediating role of artefacts (including
ICT), and offers comprehensive and flexible models to account for and
analyse the dynamics and complexity of human activity, action and the
context where it is situated. In addition, AT provides theoretical suggestions
about general principles underlying human activity that can be used to
inform and enlighten the analysis. This section begins by reviewing Vygotsky
and cultural-historical psychology that provided the foundations for AT.
Then the basic principles of AT are presented followed by a short description
of some of the methodological assumptions on which AT studies are based.
Finally the section closes with a discussion of the implications of AT for the
DoCTA project. The majority of the writings in this section are taken from
Guribye (1999), Guribye & Wasson (1999) and Mørch & Wasson (1999).
3.1
Vygotsky and Cultural-Historical Psychology
The sociocultural perspective has its theoretical heritage in the ideas of
the Russian psychologist, semiotican and pedagogical theorist Lev
Vygotsky and his followers (Wertsch, del Río & Alvarez, 1995). This
research tradition, referred to as Cultural-Historical Psychology6 ,
appeared in the former Soviet Union in the nineteen twenties and thirties.
Vygotsky, his students and his colleagues were committed to formulating
a psychology grounded in the ideas of Marx and Engels (Cole & Scribner,
1978). Thus, the task at hand was to create a theory that was a
“psychologically relevant application of dialectical and historical
materialism (Ibid, p. 6)”. In formulating a cultural-historical theory of
6
42
The term ‘Cultural-Historical Psychology’ is used when referring to the heritage
form Vygotsky, Leontev, Lauria and other Soviet psychologists. When dealing
with the appropriation of this tradition in contemporary work and debates
(especially in the West) the term ‘Sociocultural’ is applied (See Wertsch, del Río
& Alvarez, 1995; Zinchenko, 1995).
Chapter 3 Conceptual framework for evaluations
higher mental processes, Vygotsky rejected the dominant perspectives
and traditions (e.g., behaviourism) in psychology at his time (Vygotsky,
1978; see also Minick, 1996). A brief presentation of some fundamental
concepts in the Vygotskian framework is given in the following subsections
Internalisation
A central tenet in this framework is the social nature of individual
development, including higher mental functioning. According to Vygotsky
many mental functions appear on two planes; first interpersonally (or
inter-psychological) on a social level through social interaction, and later
intrapersonally (or intra-psychological) on an individual level (Vygotsky,
1978). This is known as the genetic law of cultural development. The
process in which the linking of these levels or where “the internal
reconstruction of an external operation (Ibid. p. 56)” occurs, is called
internalisation. In light of the dichotomization of “the inner” and “the
outer” in cognitive psychology, the concept of internalisation has somehow
become a controversial one (see e.g., Marton & Booth, 1997; Rogoff,
1990). This debate seems to question whether the term internalisation
presupposes a rather strict distinction between mind and environment
and thus is subject to the ontological stance often referred to as Cartesian
Dualism. Leading theorists within the sociocultural perspective argue that
the term is compatible with a non-dualistic view, and interpret internalisation as “simultaneously a social and an individual process (John-Steiner
& Mahn, 1996, p. 10)”. This debate is not elaborated on further in this
discussion, and the internalisation process is primarily seen as a
confirmation of the view that human thinking and learning are, to a large
extent, social processes. Following Vygotsky’s genetic law of cultural
development, Säljö (1996) argues that “categories used between people
in discursive practices to account for the world (inter-psychological
categories), will appear as tools for thinking within individuals (i.e. intrapsychological categories) (p. 84)”. This brings us to the next central idea
in the Vygotskian framework, semiotic mediation.
The mediating role of artefacts
As a further elaboration on the interdependency between the social and
individual planes, Vygotsky heavily emphasises mediation and the role of
tools in human practices. These tools can be both technical and
psychological:
43
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
“The most essential feature distinguishing the psychological tool
from the technical tool, is that it directs the mind and behaviour
whereas the technical tool, which is also inserted as an intermediate
link between human activity and the external object, is directed
toward producing one or other set of changes in the object itself
(Vygotsky, quoted in Daniels, 1996, p. 7)”.
The concept of mediation reflects that human action, on both individual
and social planes, is mediated by tools and signs (the latter corresponds
to psychological tools) (John-Steiner & Mahn, 1996). In other words,
“tools serve as mediational means, i.e. they – metaphorically speaking –
stand between the individual and the world (Säljö 1996, p 84)”. This
process is depicted in figure 2.
Figure 2
Vygotsky’s mediating triangle (Vygotsky, 1978)
Opposing Pavlov’s stimulus-response model of human behaviour, Vygotsky
introduced a mediating link between the stimulus and its response (ÅbergBengtsson, 1998). Though, this should not be interpreted as a mere
extension of Pavlov’s model. All the higher psychological processes are
mediated through a tool (i.e. via X in figure 2), but in a “much more
sophisticated form than that shown in the figure (Vygotsky, 1978, p.
40)”. One of the most important psychological tools is language, which
serves as the “prime device for rendering the world intelligible and for
communicating our intentions to others (Säljö, 1996, p. 84)”. Other
examples of mediational tools include: systems for counting; algebraic
44
Chapter 3 Conceptual framework for evaluations
symbol systems; diagrams; maps; and a more recent invention, the computer (John-Steiner & Mahn, 1996).
More generally, the theme of mediation as formulated by Vygotsky plays
an important role in the sociocultural perspective in that it provides “the
link or bridge between concrete actions carried out by individuals and
groups, on the one hand, and cultural, institutional, and historical settings,
on the other (Wertsch, del Río & Alvarez, 1995, p. 21)”.
Zone of proximal development
Discussing the relation between learning and development, Vygotsky introduced
the concept zone of proximal development (ZPD), which he described as:
“the distance between actual developmental level as determined by independent problem solving and the level of potential development as
determined through problem solving under adult guidance or in
collaboration with more capable peers (Vygotsky, 1978, p. 86)”.
This concept helps to explain the way that social and participatory learning
takes place (John-Steiner & Mahn, 1996), and can be seen as an argument for collaborative learning. Wells (1996) claims that in light of the
ZPD, both the teacher’s role in learning and the role of collaboration in
learning are viewed in a new and different way. Since Vygotsky first
applied this concept, many different interpretations have been suggested (see e.g. Lave & Wenger, 1991; 1996). As this is not a central
concept (except as a theoretical foundation of collaborative learning) for
the analysis that is presented in this research, it will not be elaborated
any further.
Having briefly presented some of the essential ideas of Vygotsky and
cultural-historical psychology, focus is now turned to his student and
colleague A. N. Leontev who played a crucial role in the development of
activity theory as it is known today.
3.2
Basic principles of Activity Theory
In the following paragraphs, some of the ideas that constitute the body of
knowledge referred to as activity theory (AT) are presented. Activity theory
45
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
emerged from cultural-historical psychology7 where, as pointed out
above, Vygotsky was a central figure. Vygotsky himself never explicitly
examined the concept of activity, but strongly influenced the development
of activity theory (Wertsch, 1981).
As Kuutti (1996) points out, activity theory is not a theory per se, rather
it is “a philosophical and cross-disciplinary framework for studying different forms of human practices as developmental processes, with both
individual and social levels interlinked at the same time (p. 25)”. The
main interest in AT in this project is that it provides a “powerful
sociocultural lens through which we can analyse most forms of human
activity (Jonassen & Rohrer-Murphy, 1999, p. 2)”, and that it is suggested as an alternative framework for HCI research (Nardi, 1996),
including CSCW and CSCL.
Unit of analysis
A central tenet in the AT framework is that of unity of consciousness and
activity (Kaptelinin, 1996). The idea is that it is through interaction with
the environment that the consciousness or the mind as a whole emerges
and comes into existence. This interaction, i.e. the activity, is socially and
culturally determined. To understand human praxis and consciousness
(e.g. thinking and learning) the proper unit of analysis, according to AT,
is activity. The concept of activity always includes a minimal meaningful
context for understanding individual actions (Kuutti, 1996). According to
Engeström (1987), activity “is the smallest and most simple unit that still
preserves the essential unity and integral quality behind any human activity
(p. 81)”. In focusing on activity as the basic unit of analysis, emphasis is
put on the cultural, institutional and social settings in which these activities occur. One can thus argue that AT also provides the necessary
conceptual resources for capturing essential elements of these settings.
The activity system
A common reformulation of Vygotsky’s mediational triangle (as depicted
in figure 2) is shown in figure 3.
7
46
See Zinchenko (1995) for a further treatment of the relation between culturalhistorical psychology and the theory of activity.
Chapter 3 Conceptual framework for evaluations
Figure 3
The basic mediational triangle (Cole & Engeström, 1993)
As Cole & Engeström (1993) explain, this model “although useful as
schematic ‘minimal structures’ of human cognitive functions”, fails to
“account for the collective nature of human activities (p. 7)”. According
to Leontev (1978) an activity, or an activity system as he called it, consists
of a subject, the object of the activity, and the community in which the
subject is constituted, thus stressing the collective nature of activity. The
relations between these three central components of an activity are
mediated in a reciprocal way (Kuutti, 1996). The relationships between
subject and community are mediated by the group’s collection of artefacts on the one hand, and by the community’s rules of interaction on the
other. The intermediate term introduced to describe the mediational
relation between community and object is ‘division of labour’ and should
be understood as “the continuously negotiated distribution of tasks, powers
and responsibilities among the participants of the activity system (Cole &
Engeström, 1993)”. The activity system, as modelled by Engeström
(1987), presented in figure 4, is an expanded version of the mediational
triangle to include the collective dimension.
47
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 4
Engeström’s (1987) model of the activity system
The model is slightly simplified for the sake of clarity. All the elements (including
the “mediators”) of the activity are actually connected, forming a systemic
whole (Kuutti, 1996). All the terms should be conceived in their broad sense:
“A tool can be anything used in the transformation process, including
both material tools and tools for thinking. Rules cover both explicit
and implicit norms, conventions, and social relations within a community. Division of labour refers to the explicit and implicit
organisation of a community as related to the transformation process
of the object into the outcome. Each of the mediating terms is
historically formed and open for further development (Kuutti, 1996,
p. 28, italics in original)”.
Activity can be understood as a form of doing, a social or cultural praxis,
and transforming an object into an outcome is what motivates an activity.
Individuals usually participate in several activities simultaneously, and
these activities are distinguished from each other according to their objects
(Kuutti, 1996).
48
Chapter 3 Conceptual framework for evaluations
Further elaborations of the ideas that animate activity theory are organised
into a set of five basic principles (Kaptelinin & Nardi, 1997). These
principles are presented separately in the following subsections, but should
be comprehended in a holistic way:
“These basic principles of activity theory should be considered as
an integrated system, because they are associated with various
aspects of the whole activity. A systematic application of one of
these principles makes it eventually necessary to engage all the
others. For instance, understanding the hierarchical structure of
an activity requires an analysis of its object or motive, as well as
developmental transformations between actions and operations,
and between internal and external components. The latter, in
turn, can critically depend on the tools used in the activity
(Kaptelinin & Nardi, 1997, p. 3, italics in original)”.
Object-orientedness
The principle of object-orientedness was identified by Leontev when he
drew the connection between an activity realised by a subject and the
objects of the external world (Stetsenko, 1990). The concept of “objects
of the external world” is not identical with the concept of “thing”:
“Whereas a thing may be regarded as any element, any aspect of
reality characterised by spatiotemporal physical determinateness,
by ‘object’ … we understand a form embodying the sociohistorical
experience of mankind. An object is the vehicle of this experience
and embodies a specific aspect of human social practice; it is the
form in which a physically defined thing functions as people go about
their life activities in society. The essence of an object functioning in
the social process is constituted not by its physical properties, but by
the specific connections and relations that become known in the
process of collective activity (Stetsenko, 1990, p. 59)”.
The notion of object in activity theory is not limited to physical, chemical
or biological properties, but in its wide sense, embracing both socially
and culturally determined properties (Kaptelinin & Nardi, 1997). As Jonassen & Rohrer-Murphy (1999) put it, “the object of an activity can be
anything, so long as it can be transformed by subjects of the activity
49
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
system. Objects might be physical objects, soft objects, or conceptual
objects (p. 8)”.
In short, an activity is oriented or directed towards an object, which is
held by the subject (a person or a group engaged in the activity). This
“object (in the sense of ‘objective’) motivates activity, giving it a specific
direction (Nardi, 1996, p. 73)”.
Hierarchical structure of activity
According to Leontev (1978) activities are organised into three
hierarchical levels. These levels are based on a very functional criteria –
what is a subordinated level (e. g. action, see below) in one activity can
be considered the entire activity in another situation. The levels Leontev
differentiated between are activities, actions and operations. Kuutti
(1996) gives a comprehensible description of these levels:
“Activities are longer-term formations; their objects are transformed into outcomes not at once but through a process that typically
consists of several steps or phases. There are also need for shorterterm processes: activities consist of actions or chains of actions,
which in turn consist of operations (p. 30)”.
People engage in activities to fulfil motives of which they may or may not
be consciously aware (Kaptelinin & Nardi, 1997). To realise these activities certain actions must be performed. These actions are directed towards
a conscious goal, and are related to one another by the same overall
objective. The activities form a frame of reference within which the individual actions can be understood. Actions, in turn, are composed of
functional sub-units called operations. These operations are automatic
processes that are routinised and unconscious. Unlike actions, they are
not directed toward a goal, but are carried out automatically, providing
an adjustment of actions to the current situation and the prevailing
conditions. Leontev offers a classical example of the action-operation
dynamics:
“When learning to drive a car, the shifting of the gears is an action
with an explicit goal that must be consciously attended to. Later, shifting gears becomes operational and can no longer be picked out as a
special goal-directed process: its goal is not picked out and discerned
50
Chapter 3 Conceptual framework for evaluations
by the driver. Conversely, an operation can become an action when
conditions impede an action’s execution through previously formed
operations (Leontev, cited in Kaptelinin & Nardi, 1997 p. 2)”.
The distinction between both operations and actions, and actions and
activities are blurred – there are no firm boarders – movements between
the hierarchical levels in both directions are possible (Kuutti, 1996). The
hierarchical structure of activity are depicted in figure 5.
Figure 5
The hierarchical structure of activity (Leontev, 1978)
Internalisation/Externalisation
Elaborating on Vygotsky’s concept of internalisation, activity theory
differentiates between internal and external aspects of activities. “The
subject and the object of an activity are in a reciprocal relationship with
each other: the subject is transforming the object, while the properties of
the object penetrates into the subject and transform him or her (Kuutti,
1996, p. 32)”. In this internalisation, as Leontev notes, “processes are
subjected to a specific transformation: they are generalised, verbalised,
abbreviated and, most importantly, become susceptible of further
development which exceeds the possibility of external activity (Leontev,
cited in Kuutti, 1996, pp. 32-33)”. According to AT, the internal “mental”
processes can not be understood without reference to external processes,
and thus AT rejects the dualistic concept of an isolated, independent “mind”
(Kuutti, 1996). The internal processes are formed at a social level in
collaboration and interaction with others (or external artefacts). The other
way around, “mental processes manifest themselves in external actions
performed by a person, so they can be verified and corrected, if necessary
51
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
(Kaptelinin, 1996, p. 109)”. Thus, externalisation is important in
collaboration between people. In order to co-ordinate their activities, they
must be performed externally (Kaptelinin & Nardi, 1997).
Artefacts and mediation
As stressed by Vygotsky (1978), artefacts mediate or alter the nature of human activities. All activities contain different artefacts, and these artefacts or
tools embody a certain history and culture. “Artefacts themselves have been
created and transformed during the development of the activity itself and
carry with them a particular culture – a historical residue of that development
(Kuutti, 1996, p. 26)”. Thus, artefacts are carriers of cultural knowledge and
social experience. This experience and knowledge is manifested in both the
structural properties of the tool, and in the way the tool should be used (Kaptelinin
& Nardi, 1997). The structural properties are linked to Gibson’s (1979) term
affordances that “refers to the perceived and actual properties of a thing,
primarily those functional properties that determine just how the thing could
possibly be used (Pea, 1993, p. 51)”. Activity theory stresses that an artefact
first comes fully into being when it is actually used. Hence, knowing how to use
it is an essential part of the artefact (Kaptelinin & Nardi, 1997).
The concept of tool mediation implies an asymmetrical model of people
and things (e.g. the computer) unlike that of cognitive science (Nardi,
1996), and at the same time invites a serious study of how artefacts are
an inseparable and integral part of human activities.
Development
Activity theory perceives activities not as given or static, but as dynamic processes
under continuous change and development (Kuutti, 1996). Thus, to fully understand an activity, its history and development should be taken into
consideration. This development occurs at all the different levels in the activity:
“New operations are formed from previous actions as participants’ skills
increase; correspondingly, at the level of actions the scope of new actions
is enlarging, and totally new actions are being invented, experimented
with, and adapted as responses to new situations or possibilities
encountered in the process of transforming the object. Finally, at the
level of activity the object/motive itself (and the whole structure of activity
52
Chapter 3 Conceptual framework for evaluations
related to it) is reflected, questioned, and perhaps adapted, reacting to
larger changes and other activities (Kuutti, 1996, pp. 33-34)”.
In addition, all the elements of an activity are subject to continuous development.
These dynamics underlying the activity are represented in figure 6.
Figure 6
The nested nature of activity dynamics (Engeström 1987)
Activities are not isolated units, “but more like nodes in crossing hierarchies
and networks, they are influenced by other activities and other changes
in their environments (Kuutti, 1996, p. 34)”. This also emphasises the
collective nature of activities, and Engeström describes these activities as
“systems of collaborative human practice (Engeström, 1987) ”.
A major driving force in the development and change of activities are,
according to Engeström (1987; 1991) contradictions. These multileveled contradictions can exist both between and within activities, and will
“manifest themselves as problems, ruptures, breakdowns, clashes (Kuutti,
1996, p. 34)”.
Viewing activities as developing processes also reveals certain
methodological implications. For example, in order to understand how
53
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
an artefact is used, one has to study its use over time allowing for the
usage to develop (Kaptelinin & Nardi, 1997) and not for example in a
single one-time laboratory experiment.
3.3
Methodological Issues
Having discussed the five basic principles of activity theory, what remains
is to take a closer look at some of the methodological assumptions on
which activity theory studies are based. Vygotsky writes:
“The search for methods becomes one of the most important
problems of the entire enterprise of understanding the uniquely
human forms of psychological activity. In this case, the method
is simultaneously prerequisite and product, the tool and the
result of the study (Vygotsky, 1978, p. 65, italics in original)”.
One of the most important methodological commitments of
contemporary8 activity theory studies are to examine activities in situ –
in the environment where it “naturally” occurs. Nardi (1996) summarises the methodological approach in activity theory into a set of
characteristics (p. 95):
• A research time frame long enough to grasp the objects of the activity.
• Attention to, first, broad patterns of activity, then (within the context
of the activity as a whole) more narrow episodic fragments can be
analysed.
• The use of a varied set of data collection techniques including interviews,
observations, video and historical materials.
• A commitment to understanding things from the users’ points of view.
3.4
Some general remarks on AT
Activity theory was mainly developed within the confines of the former
Soviet Union, and has only more recently been increasingly adapted by
western (Anglo-American) researchers (see e.g., Wertsch, 1998;
8
54
See Cole & Engeström (1993) for an account of the methodological assumptions
of Vygotsky, Leontev and Lauria.
Chapter 3 Conceptual framework for evaluations
Engeström, Miettinen & Punamäki, 1999). Some research traditions in
the west have followed “similar avenues of thought (Kuutti, 1996, p.
25)”. It is possible, for example, to trace parallels between AT and Dewey’s
pragmatism (Kuutti, 1996, see also Bredo, 1994) and Mead’s (1968)
symbolic interactionism (see Holzman, 1996 for an extensive treatment).
An approach closely related to AT is that of Distributed Cognition (cf.
Salomon, 1993; Hutchins, 1991; 1995). Nardi (1996) notes, rather
informally, that:
“activity theory and distributed cognition are very close in spirit …
and it is my belief that the two approaches will mutually inform,
and even merge, over time, though activity theory will continue to
probe questions of consciousness outside the purview of distributed
cognition as it is presently formulated (p. 89)”.
This is also apparent in Cole & Engeström’s (1993) article “A culturalhistorical approach to distributed cognition”, where they use AT to analyse different notions of distributed cognition.
An important feature in the distributed cognition approach is the
distribution of cognitive activities between the human mind and the environment or context in which it operates. This implies that you cannot
analyse the human cognition and the way in which it is functioning in
isolation, but that one has to see intelligence as a feature of both the
individual’s mind and the different physical and mental artefacts that are
a part of the particular cognitive phenomena of human activities (see
e.g, Pea, 1993). The focus is on the mediational means of the artefacts,
and the interaction that these artefacts mediate between individuals and
the context or environment. Hutchins (1995) argues in his book
“Cognition in the Wild” that the boundary between inner and outer,
between individual and context, should be softened. According to Hutchins,
a more fruitful unit of analysis than the cognitive system of an isolated
individual, is to focus on the individual located and actively engaged in a
culturally constructed world ¾ thus, leaving us with only a subtle difference
in respect to activity theory.
55
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
3.5
Activity theory and design
Our experience (Mørch & Wasson, 1999) with AT is that it provides a
lens through which we can see the world. It is useful to orient thoughts
and research questions. It provides a number of methods/tools shaped
by a general theoretical approach. It can be used to guide methodological
decisions regarding evaluation (how technology is used) and to a lesser
extent regarding design (how technology will be used). In fact, from an
Activity Theory perspective we should always talk about redesign because we are “redesigning” work/learning praxis.
Adoption of these approaches provides a strong and fruitful conceptual
framework that informs both the design and evaluation of collaborative
telelearning scenarios. In order to study human activity and evolution of technology in DoCTA, we take as a starting point the “use of technology” and not
the technology per se (Mørch & Wasson, 1999). Our working hypothesis is
that the interplay between change of technology and change in activity (through
the use of technology) is significant. We thus design our case studies to explore
this interplay in various ways. Activity theory reminds us to always focus on
context and suggests the articulation of the design of artefacts as “change in
activity”. From this perspective activities evolve and change and artefacts
evolve and change, thus mutually influencing each - but not in the same way.
To operationalise this view and to articulate and distinguish activities and artefacts we integrate Activity theory with a design perspective. We see artefact
evolution as continual redesign of ready-made application units (Norman, 1990;
Mørch, 1995). Furthermore, the evolution of these artefacts needs to be
informed by the practice carried out by users when they use the artefacts in
genuine human activity. Otherwise, the redesign will be of little practical value.
The rationale for this is that evolution of artefacts needs to be informed
by the practice carried out by users when they use the artefacts in genuine
human activity. Otherwise the redesign will be of little practical value.
The diagram in figure 7 is an attempt to put Activity Theory in such a
context in order to help us focus our discussion.
56
Chapter 3 Conceptual framework for evaluations
Figure 7
Integration of a design perspective with Activity Theory
(Mørch & Wasson, 1999)
From evolution of artefacts to meaningful tasks (and
back again)
Figure 7 can be seen as an extension of Carroll and Rosson’s task-artefact
cycle developed in HCI (Carroll et al., 1991). The task-artefact cycle
describes the interdependence of tasks and artefacts in human activities.
There is a reciprocal relationship between the two. When tasks change,
artefacts tend to change as well. Carroll and Rosson give as an example
the evolution of writing tools. When the typewriter was replaced by the
word processor this was not only an evolutionary change in technology
— it also changed the task of writing. For example, with a word processor
it is now possible to correct a misspelled word without the reader ever
knowing that it was once misspelled. What consequences this has for the
evolution of groupware and whether or not we can find similar or other
kinds of relationships between the other components in the diagram is
something we are exploring.
57
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
58
Chapter 4 Evaluation of Groupware Systems
4 Evaluation of Groupware
Systems
HEGE HIGRAFF
The focus in this thesis (Higraff, in preparation) has been on the evaluation
of four different groupware systems for use in collaborative telelearning.
The evaluation has addressed usability, the interface, and the relative
usefulness of each system. The four groupware systems are: TeamWave
Workplace (TW), Lotus LearningSpace (LearningSpace), Habanero (H)
and Basic Support for Co-operative Work (BSCW). Each system was
used by a geographically dispersed pair of students9 to carry out a
collaborative telelearning task. Since these groupware systems have very
different capabilities, it was quite difficult to find a task that could serve
as a benchmark. The four student pairs were asked to design UML
diagrams for a dental office information system (this will be explained in
more detail in the section on the scenario design). Since two of the systems did not offer a drawing tool, however, the student pairs collaborating
through LearningSpace and BSCW used Microsoft Paint to draw their
diagrams. The diagrams could then be shared through the system (e.g.,
by posting the diagrams as messages in BSCW).
4.1
Overview of the four groupware systems
Habanero
Habanero is a framework developed by NCSA (National Center for
Supercomputing Applications), University of Illinois, to help developers
design shared applications in an easier fashion than building from scratch.
9
The 8 volunteer were undergraduate students from the Department of Information Science at the University of Bergen. They had no experience with any of
the systems.
59
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The framework is supposed to help the users to reuse existing infrastructure for communication and toolkit development. The programmers
will be able to develop new applications from examples, or to develop
new ones from the application framework. The framework itself provides
the co-operation functions (Chabert et. al, 1996).
The goal for Habanero is “to provide a cross-platform, generic and
extensible synchronous collaborative framework to cover a wide range
of collaborative functionality’s for different user communities. (Chabert,
1997, p. x)”. Habanero supports only synchronous co-operation.. It is
therefore primarily a real-time system where everybody collaborates at
the same time. It does not support asynchronous tasks.
Habanero is implemented in Java, which is supposed to make it platformindependent. Also the fact that Java is object-oriented gives room for reuse and code-inheritance. The fact that Java is object-oriented is not
important unless one actually has the source-code, and this is understandable primarily to programmers. Users of the applications created
with Habanero will typically not be interested in the source code.
The original tools include (NCSA Habanero, 1998):
• Whiteboard — drawing-tool
• Telnet — for remote login on different machines
• Savina — web browser
• Voting Tool — to vote for different proposals
• Chat — text-based chat
• Audio Chat — voice-based chat (only for Solaris on Sparc-machines,
for now)
• mpEdit — viewing and editing text-files
During the summer of 1999, four new tools have been. These are:
• Habanero Neighbourhood — distributed file-system for file-sharing
• ClipNShip — download and use other user’s files, without leaving the
session
• The GIS Viewer — viewing and manipulating geographic information
• The Virtual Network Computing (VNC) — allows remote control of
any system running a VNC server
Habanero is an application framework — not a complete application.
Applications are built by extending the framework by adding application60
Chapter 4 Evaluation of Groupware Systems
specific subclasses. One can develop new tools on top of this framework,
and one can develop new tools from scratch. There are also examples of
existing tools that can be reused and modified. The tools can also be
removed, if one finds them of little value. Thus, Habanero can easily be
adapted to different purposes. For the purposes of this evaluation study,
Habanero is tailored for individual use.
LOTUS LearningSpace
Lotus LearningSpace (LearningSpace) is a groupware system developed
by Lotus Development Corporation. It consists of a platform with different
programs attached to it. One can work synchronously through
LearningSpace Live, or asynchronously through LearningSpace Forum.
A combination of the two Lotus products would give the most flexible
environment. The educational goal of LearningSpace is that learning should
be possible anytime and anywhere, and they aim to produce products that
are innovative, responsive, productive, and competent (LDC, 1998a).
In LearningSpace it is possible to do assignments, run tests and exams,
as well as collaborate on different levels. The groupware system is flexible
enough for people who would like to co-operate with others, but who do
not have the time to participate in an ongoing collaboration which demands
100% real-time co-operation. LearningSpace also provides a kind of
database in which one can post messages, articles etc.
Lotus LearningSpace comprises the following tools:
• Schedule — course-outline and tasks from the instructor
• MediaCenter — all material for course and tasks
• CourseRoom — interactive collaboration environment
• Profiles — “homepage” for each student
• Assessment Manager — instructors “home-directory”
• Application Sharing — instructor can fetch windows-applications
• IP Audio and Video — communication-means for use in the ”classroom”
It is also possible to do adapt LearningSpace. Two supplemental tool packages
are provided for developers and administrators (LDC, 1998b). These are:
• Customise — tools to modify the look and feel of LearningSpace and
to build custom functions and modules. New features provide unique
customisation for different courses on a single LearningSpace server.
61
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
• Central — tools to manage the creation, installation, configuration
and student enrolment/access processes of LearningSpace courses.
In this evaluation study, only LearningSpace Live, the synchronous environment, will be used.
BSCW Shared Workspace system
BSCW, developed by GMD (German National Research Center for Information Technology), is a groupware system which can either run on a
local server, on the web, or on the producer’s (GMD) server.
BSCW enables collaboration over the Internet. It is a shared work environment that supports uploading of documents, notification of events
(such as arrival of new messages), group-management, etc. The only
software needed for using the system is a standard web-browser (GMD,
1999).
BSCW is built on the “shared workspace” metaphor. This means that the
user has the opportunity to save information in such a way that other
members of the group also have access to it. This means each group
member has the same permissions to edit, post or delete any document.
In many ways it is similar to Lotus LearningSpace. In both groupware
systems you post messages on a page (either on web or on a local server)
for everybody to read.
The primary goal of the BSCW project was to construct a platform that
provides the elements needed for collaboration in geographically dispersed groups. They attempt to do this independently of the different
infrastructures the users have on their desktop, network or hardware.
The project focuses on how to help users collaborate on writing documents.
To this end the system supports version-control and both new and old
version of the documents are accessible. To prevent several people editing a document at the same time, a lock mechanism is provided (Bentley
et. al, 1997). BSCW does not support chating or drawing/painting.
Since the task performed in this evaluation study required drawing, the
student pair was allowed to use MS Paint in addition to BSCW.
62
Chapter 4 Evaluation of Groupware Systems
TeamWave Workplace
TeamWave Workplace (TW) is the groupware system based on a room
metaphor. It supports both synchronous and asynchronous communication
between the participants. According to the developers, TW is a CSCW
(computer supported co-operative work) system “designed to help people
work together. It is versatile enough to support telecommuting and telework, distance learning, case-based learning, and corporate training
(TeamWave Workplace, 1999)”. TW runs on a variety of platforms
including Macintosh, Windows95 and Unix (AIX, Sun, Linux).
In TW rooms are created for different purposes. Rooms can be created
by the instructor or by the participants themselves. A group might create
a room where they will collaborate on a task, or an instructor might
create a room where assignments can be left and messages can be posted.
Access to each room is regulated by its creator. For example, there
could be individual access (to a single user), group access (for a defined
group of users), or access for everyone.
TW provides the participants with the following tools:
• Address Book — shared address book
• Brainstormer — to generate ideas; ideas are posted in the same
sequence as they are suggested
• Calendar — shared calendar for making appointments
• Concept Map —organising concepts and ideas in a visual interface
• Database — a simple database
• Doorway — shortcut connection to the other rooms on the server
• Fileholder —shared file “container”; files can be stored, collected and
opened by anyone
• File Viewer — view text simultaneously with other participants
• Image Whiteboard — pictures can be opened and edited
• Meeting Roster — to arrange meetings
• Message Board — newsgroup-like discussions; placed on the
Whiteboard
• Postit — like yellow post-it notes for exchanging messages and leaving reminders
• Slide Tool — slide-show with slides made in MS PowerPoint
• To Do List — list of what have to be done
• URL Reference — direct link to a web pages
• Vote — to cast a vote in a discussion
• Web Browser — to view HTML pages with others
63
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
TW has been used in several graduate courses in pedagogical information science at the University of Bergen. In this evaluation study, a task
room was made for the student pair.
4.2
Test Scenario Design
The test scenario was completed over a three day period in October 1999.
This section describes the task, the physical setting and the data collected.
The task. The volunteers students were given a system modelling task
that was connected to an undergraduate course (grunnfag) they were
taking. The task (see Appendix C) required that they design a use-case
diagram and a class-diagram for an information system to be used in a
Dental Centre. Each pair was given a short introduction to the groupware
system they were to use. Then they were given the task assignment and
could begin to work, each at their assigned computer.
The physical setting. To simulate geographical distance between the
collaborating students, two computers10 were placed different floors of
IFI. One of the students in the pair was video taped and observed while
working on the assignment. Figure 8 illustrates the placement of the
computer screen, student, camera and observer.
Data-collection. During the collaboration task, a video camera was used
to record what one of the students said and what they did on the screen
(see figure 8 for camera placement). This data will give us an impression
of how easy it was to learn the different systems and give us an idea of how
long it took them to feel comfortable with the systems. The sessions were
relatively short, lasting between one and three hours. The students were
also interviewed (see interview guide in Appendix D) after the sessions.
Two of the groupware systems created log files and screen-cam and screenshots were used to save pictorial information about the sessions.
After the three days of the scenario, there is about 6 hours of videotape, 8
interviews and miscellaneous screen-shots, event logs etc. The video is currently
being transcribed and analysed. The full results will be reported in (Higraff,
in preparation). A preliminary analysis, however, is presented below.
64
Chapter 4 Evaluation of Groupware Systems
Problems during the test scenario. One day the network crashed
causing a major problem since the students were not able to connected
anymore. This also affected the tests performed the next day. In these two
cases the students were not able to complete their assignment — they
had, however, completed a high enough percentage of it to warrant inclusion. Both the diagrams were almost finished, and they had shown if and
how they were able to do such a task in the different groupware systems.
Looking at what they had completed, we decided not to redo the tests.
Figure 8
The physical setting of the scenario
65
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
4.3 Preliminary Analysis
Lotus LearningSpace
This was the first groupware system to be tested. The two students with
respect to how fast they started to try the system. Student B started
experimenting immediately, while A began by reading the assignment.
This resulted in A having problems following B in solving the task.
The following11 is a comment from A to B, regarding A having problems
following B:
A: “Takes time to understand this- I hope I soon catch up regarding
drawing the diagrams”
B: ”Hang in there! J”
After getting going, the students quickly agreed to each make a solution and post
it in LearningSpace Live. Working in this manner, it was more like doing the work
twice than dividing the work among them selves. Over time, their work style
shifted more to A commenting the work B had posted, and then B commenting
on what A said or just correcting it, if he believed his was a better proposal.
The students finished their assignment in about two hours. During this time
they came up with proposals, complaints and positive comments to
LearningSpace Live. B uttered a wish for having a shared whiteboard, which
LearningSpace Live did not have. He also wanted to use Rational Rose12 as
the drawing tool, and he wanted to have a chat tool.
TeamWave Workplace
In preparation for this scenario a new room on the TeamWave server
was created. A concept map with some ready-made icons was placed in
the room. We also posted a post-it note with information about the task.
It took them approximately 1 hour and 30 minutes to complete the
assignment. They had access to all the tools in the program. Having
been told up front, however, which TW tools were suitable for drawing,
they focused on those tools. They also used the chat tool extensively.
11
12
66
This and subsequent dialogues have been translated from Norwegian to English
by the author.
Rational Rose is a tool for object-oriented system design used in the
undergraduate (grunnfag) course. It provides support for UML-notation and
Booch-notation that simplifies designing object-oriented systems.
Chapter 4 Evaluation of Groupware Systems
During the first few minutes, they spent time reading. Then, they used
the chat-tool to make proposals before they posted anything. After a
while this pattern changed. They posted, waited for a reaction from their
partner, and then continued if they did not hear anything.
The following discussion shows their co-operation in the beginning:
A: [12:55:25] bruker1 says: Then we have to start renaming the
use cases.
B: [12:55:32] bruker2 says: What use cases do we have?
A: [12:56:10] bruker1 says: What about “Making Appointment”?
B: [12:57:28] bruker2 says: I agree. But one can make an
appointment through both dentist and
assistant. – shall we make sub-cases?
A: [12:59:10] bruker1 says: Let us decide that making
appointments can only be done
through a secretary.
B: [12:59:30] bruker2 says: OK
The following comments appeared a bit later in the scenario:
A: [13:10:33] bruker1 says: I have done a relocation as you can
see. Agree?
B: [13:10:51] bruker2 says: Ditto! I have made connection “report-dentist”, OK?
A: [13:11:01] bruker1 says: Yes
As can be seen from the above dialog, the pattern in their comments
changed over time. It started with one student giving approval to the other’s
proposals, and ended with commenting and encouraging each other etc.
The two of them did not have much difficulty collaborating, and also it
seemed as if they knew how the other student wanted things to be done.
Habanero
The two students using Habanero, first read through the assignment before
they contacted each other on chat. Both of them co-operated well, but
expressed frustration regarding functionality in the groupware system.
Habanero is not a program, but a framework with demo tools on it, and
it still lacks when it comes to functionality. The students agreed, however,
that it was easy to use, and it had a very simple user interface.
67
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
They decided to open two whiteboards and alternated between them,
see figure 9. They drew a use case-diagram on one of them and the
other was used for scratch paper and to create the class-diagram. They
divided the tasks between themselves as illustrated in this dialogue:
A: Hege>
Ok. I try to draw, and you list the entities...
B: Habanero> Is it possible to use the function “ copy”, to get
more stickmen13 ?
A: Hege>
Don’t know. Let’s try.
B: Habanero> Does the list look ok?
A: Hege>
Excellent!
A: Hege>
Have you managed to copy anything?
B: Habanero> No, I’m trying to find the help-program to get some
hints…
B: Habanero> I can’t find anything. Do you want me to draw too?
A: Hege>
Sure, just get going..
Figure 9
13
68
The two white boards used by the students in Habanero
The stick-men had to be made in MS Paint.
Chapter 4 Evaluation of Groupware Systems
Their strategy, having two whiteboards running at the same time, worked
until they started to rearrange the diagrams. They found it impossible to
use copy & paste like they were used to, which was, according to them,
a major drawback. After this the subjects experienced a network
breakdown before having completed their assignment. As they had nearly
completed the task, we did not rerun the session.
BSCW
This groupware system contains no chat tool. To communicate, one posts
messages and texts in a database, and uses a web browser as a user
interface to the database. This caused some minor delays and a problem
of having to reload the page for updates14 .
The two students testing this system quickly decided to split the assignment,
and carry out one part each. The diagrams were made in Paint, and
exported into BSCW. This worked fine for one of the students, but the
other one had problems exporting the documents. She could write
messages but could not add her documents. I believe this was caused by
a fault in the set-up. We were not able to correct this. The problem was
partly solved by one subject still being able to post drawings, and the
other subject describing what work she had done in her messages. This
problem could easily be solved by the mail-program they had available.
The students were able to complete the assignments, but it was obvious
that there were many things to learn and too little time. They had
problems communicating in the beginning; the messages were hidden
behind different icons.
14
It is also possible to run the system on a local server.
69
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 10
Part of the class-diagram the students made in MS paint
while testing BSCW
This is a part of the conversation, taking place through mail messages
while creating, with MS Paint, the diagram shown in figure 10:
A: from rose, 06 Oct
I have made a start on the classdiagram, but I’m not able to send it. I
have Employee as a super-class, with
four sub-classes
Clients have connection to employee?
Or is it clinic?
B: svar from bruker, 06 Oct Sounds OK, client ought to be
connected to employee, not the clinic?
Am working on the use case-diagram, I
will soon send you a proposal.
BSCW is a complex groupware system with a number of opportunities.
This results in a high usability threshold when learning the different
functions of the groupware systems. Major parts of the groupware system are quite intuitive if one has used Windows earlier. One cannot,
however, make full use of the system without proper training.
70
Chapter 4 Evaluation of Groupware Systems
4.4
Discussion
The groupware systems
If one tries to compare these groupware systems, the first noticeable
difference between them is the user-interface. BSCW and LearningSpace
are both arranged with one or more pages in which to post messages. In
addition, LearningSpace has three other pages or rooms. TW is based
on a room-metaphor, and provides a lot of tools. Habanero is a framework
with few tools available, but it is organised on a desktop, a metaphor
with which most people are familiar.
Habanero is not very time-consuming to learn because it is not a fully
developed groupware system. To develop this groupware system further,
will be at least as time-consuming as learning the other ones. It also
requires a well-trained programmer to add the functions needed. Not
being entirely completed means that adjustments can be made. One can
get a groupware system for each special situation. But it will be time and
resource consuming.
Generally, the more positive comments were directed towards TW. It
was rated as user-friendly and its tools are well arranged. The students
using TW only took advantage of a few of the many tools available. I
believe they did not realise how complex the groupware system actually
was. TW can be used for both synchronous and asynchronous work. This
is an advantage if one wants to solve different assignments with such a
groupware system. In a pure synchronous system, one is dependent on
all the students being able to work on the task simultaneously. For
example, with Habanero, what is done in a Whiteboard is not saved and
the next person using the program can not see that anything has been
done. Working synchronously is not always possible. Many students choose
an on-line course believing it is possible to combine them with work or
children. And it is possible, as long as the technology supports
asynchronous work.
BSCW and LearningSpace are both primarily designed for asynchronous
work, which was not explicitly tested in the scenario. These two groupware
systems are therefore suitable for long lasting group tasks. BSCW would
be excellent for people writing a paper together. They can write and
immediately post their proposals, and the other persons can see when
somebody last edited a piece of text. LearningSpace is suitable for students
71
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
working together for a relatively short period of time, and also for teachers
wanting to run an entire course over the net. It is possible to post messages
to a group, or a person, and one can also arrange exams with this system. The fact that there are four rooms makes it fairly well arranged, as
long as the students are told what to post in the various rooms.
The groupware system causing the students the most trouble was
Habanero. As already stated, it is not a fully developed system, but a
framework with a relatively small number of tools. Habanero can only be
used for synchronous work. This can be a problem for many students
working part-time or having other obligations. The framework has
potential, but it would demand quite a lot of hard work by a good programmer to tailor it for a specific situation. One can ask if it is worth it, as
long as there are free manufactured products within reach.
Preliminary observations
One surprise with this study was the relatively low expectations most of
the students had for such a system. Even though they are information
science students, just the fact that they were able to work together seemed
to please and fascinate them.
It appears that working together online is much easier if one have access to
a chat-tool, where one can see what everybody else writes as they write it.
Email is different in that one can not see what the other person has written
until the whole message is posted. To post messages is time-consuming and
it might lead to a lot of double work. If one can read while the other types,
one will be able to see at once what the other person is trying to do.
The students started off trying the groupware system they were assigned
to use, feeling their way around. After a while, they seemed to develop
a certain work-pattern. This is not to imply that they adopted the same
pattern, but they changed the way they initially worked. In several of the
cases it seemed they became more secure in what they was doing. Instead
of having every move confirmed by the other person, they started to do
things or draw what they believed to be correct. It was impressive how
fast they seemed to adapt to this new way of co-operation.
It was also fascinating to see how they started out very formal, but after a
few minutes they began to use quite informal language. Several of them
72
Chapter 4 Evaluation of Groupware Systems
switched to dialect15 after a short period of formal orthography. It seemed
they wanted to “test” the person with whom they worked, and when this
person was found OK, they could be informal. The following extracts give
an example of this (in this example and the next example, the original
Norwegian dialect is given in italics with an English translation in parentheses).
This dialog is extracted from the first part of the session. The two are
“getting to know” each other:
[12:52:40] bruker1 says:
[12:53:15] bruker2 says:
[12:54:45] bruker1 says:
Skal jeg lage flere aktører? Tannlege osv?
(Shall I make more actors? Dentist, etc.?)
Greit (OK)
Enig i at vi har alle ansatte på høyresiden? (Agree that we have all the
employees on the right?)
The next excerpt is taken from the dialog about half an hour later. The
two seems to feel more comfortable with each other. They both use slang
or “dialect”. Also, they both use abbreviations and uncompleted sentences:
[13:31:39] bruker1 says:
[13:31:48] bruker2 says:
[13:31:51] bruker1 says:
[13:32:14] bruker1 says:
[13:32:21] bruker2 says:
[13:33:23] bruker1 says:
[13:33:49] bruker2 says:
[13:34:08] bruker2 says:
[13:34:35] bruker1 says:
15
Se! (Look!) (Subject points at new
connection-arrow)
Kossen? (How?)
Vi har laga Inheritance-pil! (We made
an inheritance-relation!)
Den ligg nederst i “tabellen” av linktyper. (It’s in the bottom of the “table”
of link-types)
Aha! (Aha!)
Ka no då? (What now?)
Forbindelsar... (connections...)
Ml.klient - tannlege? (bet. Client- Dentist?)
Ja, selvfølgelig... E litt sein i hodet i
dag... (Yes, of course…My head works a
bit slow today…)
In Norwegian there are 2 formal languages and an unknown number of dialects.
Every little place in Norway has their own dialect. The two students in this
example come from somewhere outside Bergen (on the western coast) and
somewhere close to Bodø (in the north) – their dialects are very different!
73
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
It seemed that although the students learned to use the groupware systems quite fast, they were also learning about how to communicate with
one another. This is a consequence that is not directly related to using
groupware systems. In the sessions the students found the basic tools
needed for their task, but they never got to try the whole groupware
system. Thus, the students would have benefited from having had longer
training in the different groupware systems. In VisArt, the students were
given extensive training in both use of the system and in collaboration.
Preliminary suggests for what a groupware system
should contain
The user-interface is extremely important as it gives a first impression,
and gives access to the functionality of the underlying system. If the interface is bad, it will be harder to make use of the system’s functionality. It
would be beneficial, if the interface were changeable for different tasks.
In that way one could emphasise the more important features needed
for the different tasks. In some ways TW allows the students to configure
the interface (e.g., their room) with the tools they need. In the test scenario described here, a large number of the systems’ interface features
were not needed. A preliminary impression is that the most important
tools are:
• Chat
• Drawing-tool
• Text-editor
All of the students commented at some point on the need to talk to their
partner. When working asynchronously, it is important to post messages,
but when working synchronously, time becomes an important aspect.
One should get the message instantaneously. An easy way to do this is by
a chat-like tool. The volunteers gave different reasons for why they wanted
to be able to talk more or less directly to their partner, but the reason
more often stated was time-saving needs.
We also found that a whiteboard or a drawing-tool was necessary. Of
course it is possible to use a tool outside of the groupware system, however,
one of the important reasons for using such a groupware system is that
one has all-important features collected in one place. Another important
tool to have, is some kind of text-editor. Most tasks contain some textual
74
Chapter 4 Evaluation of Groupware Systems
aspect, which one would like to edit. If such a tool is not included, one
might consider using an existing program, like Microsoft Word or Word
Perfect. In many cases, this will be less costly than making a new tool, as
a text-editor is quite complex. Also, most people are familiar with these
tools. But, as mentioned before, this must be evaluated in each specific
case. In order to evaluate whether to use an existing tool or to build an
integrated tool, one needs to find answer to the following questions:
• What will be more time-consuming?
• What will be more expensive?
• Who is the user?
Each of these tools must be fully functional. For instance, the chat should
be simultaneous, and preferably with different colours indicating the different users. The drawing-tool should support basic features as copy/
paste and print. One should also have auto-figures, and if possible,
specialised symbols. The editor should provide different fonts, maybe
spell-checks etc. Another important issue here, is how specialised such a
tool should be. Is it desirable to have a tool that can do all the things
needed for a specific case, or is it more important that it is adaptable to
several tasks? An example can be the use of Rational Rose (RR) or MS
Paint for the assignment in the scenario. MS Paint was chosen because it
was more adaptable for all programs. We experienced large difficulties
when trying to copy diagrams from RR to the different systems. Paint is
also a program with which many people have experience. Also, RR is
very specialised, while Paint can be used for several tasks. One has to
consider and weigh what is more important, reuse or specialisation. One
must also weigh the cost and what purpose one has for the program.
Again, this must be considered in each specific case.
Of the four groupware systems tested, TeamWave Workplace has more
useful tools included for the kind of task performed in the scenario. It has
a fully developed text editor, a chat tool and a drawing tool. TW also has
a special tool for organising concepts and ideas in a visual interface, the
Concept Map. Thus, in our situation the students testing TW had an
advantage. Still, one can not say the other systems were not applicable
— all the students finished their assignment, but they experienced different conditions.
75
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
76
Chapter 5 Teamwave workplace
5 Teamwave workplace
BARBARA WASSON
As TW was used in both the IDEELS and VisArt scenarios, a more indepth review of it is given. This section, based on Wasson’s review of TW
in (Fjuk, Sorensen & Wasson, 1999), describes in detail this groupware
system and highlights its strengths and weaknesses.
5.1
The basis for design
TeamWave Workplace (TW) is based on the metaphor of shared
networked places. Using real life physical team rooms (Johansen, 1991)
that provide a permanent shared working place for teams as inspiration,
the notion of virtual team rooms has been adopted in TW. These virtual
team rooms provide a permanent shared space where teams distributed
over the Internet can have meetings, store documents, share URL links
to web sites, coordinate and communicate with one another and can
carry out collaborative activities such as brainstorming or participation in
a discussion forum. Each team can build a set of different rooms according
to need. For example, they might create a common project room and
resource room, and in addition, individual rooms for each team member.
A TW server maintains the operational and interactional aspects of each
collaborating team. The server acts as a communication hub, maintains
a database of registered users and a persistence repository which stores
information about rooms and their contents. This means that all materials placed in each room are stored on the server and are available for
each team member to view at anytime. Each team member downloads a
client version and connects to the team server. The server retrieves all
information shared by the team, making it available to the client. The
77
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
latest version of TW provides server-side licensing16 so all that users
need to access TW is a user account (a user name and password created
by the server administrator).
TW is created and marketed by TeamWave Software that has its roots in
the GroupLab’s (http://www.cpsc.ucalgary.ca/grouplab/) TeamRoom
project at the University of Calgary, Canada. GroupLab is one of the
leading research groups into the end-user and technical requirements of
collaborative systems and TeamWave Workplace grew out of a series of
research systems developed within the group.
One important distinction between TW and other real-time groupware
systems is that TW is based on the metaphor of a place (i.e., a room),
while most others are based on the metaphor of a meeting. This place
metaphor choice resulted in a number of room design requirements for
TW (Roseman & Greenberg, 1996):
• rooms must be long-lived, and usable by both individuals and groups
• rooms and their contents must be fully persistent
• rooms should provide interpersonal communication facilities
• generic collaboration tools should be automatically made available in
each room
• special purpose tools can be added to any rooms
• rooms must be accessible from anywhere on a network
• access to rooms must be limited to a designated group, and
• external computer & network information should be easily accessible
from within rooms
A major strength of TW is that it provides a well-integrated set of varied
collaboration tools with a good blend of real-time (synchronous) and
asynchronous communication tools which enable anytime team
collaboration. Furthermore, TW augments both existing user interaction
tools such as email, newsgroups and conferencing, and existing
conventional applications such as word processors and spreadsheets. This
can be seen as both a strength and weakness depending on one’s point
of view. As a strength it means that the user (i.e., student) can continue
to use other communication tools and applications with which they are
most comfortable and familiar, using TW for the purpose of supporting
their team interactions. As a weakness, TW does not support the sharing
16
78
Previous versions required that each client have a user account and a license
number to connect.
Chapter 5 Teamwave workplace
of applications17 such as collaborative editors for document writing, thus
synchronous collaborative activities such as co-authoring cannot easily be
carried out.
Another strength of TW’s integrated approach is that spontaneous as
well as pre-planned intra-team interactions are supported. This means
that anytime several team members are logged on simultaneously, they
can interact without setting up a pre-specified meeting time. Furthermore,
the materials left in the room when a team member leaves remain in the
room for the other team members to see when they eventually log on.
These two features are in contract to the majority of groupware systems
that only support pre-planned meetings (with a specified start/end time)
and then when the meeting is over, all materials disappear.
TW also provides an administration client to be used by whoever is
maintaining the server and/or facilitating, for example, a course or a
project. The administrator client is used to create and delete individual
accounts, assign individuals to teams/groups, as well as change access
permissions and grant administrative privileges to a particular user. A
set of tools for managing the persistence repository is also provided.
From a CSCL perspective, TW enables collaboration and supports
genuine interdependence between team members. Team members are
able to share information, meanings, thoughts, conceptions and
conclusions through their choice of operational tool objects. These thinking
tools (signs) in turn facilitate both teammate knowledge construction and
collective growth. Furthermore, the thinking tools provide a means for
thoughts to be examined, changed and elaborated upon by fellow team
members.
The collectively oriented actions are mediated by mechanisms enabling
or supporting:
• creation of a shared networked place
• workspace awareness
These are described in the subsections below.
17
The latest versions of TW, however, provide for a seamless integration with
Microsoft’s NetMeeting.
79
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Creating a shared networked place
A shared networked place is created through individual users’ production
of tool objects (artefacts) within a room. Upon starting up TW, the user is
placed in a user-specified default room. In this default room the user can
begin to work, or can either create a new room or move (enter) to another
existing room. When newly created (see figure 11), a room consists of a
blank shared white board with a pen tray where the user can select
colored pens for producing free hand drawings and a chat tool.
Figure 11
Barbara Wasson’s (empty) Room in TeamWave Workplace
A tool menu is provided for selecting tools for customising the room for
use (e.g., to share files, create a list of things to do, etc.). To move to a
new room, the user can either enter through a doorway or select a room
from the list of all room available to the group (in figure 11 see the list of
rooms — Geir Moen’s Fast Room, Classroom, Team11 — which are
visible and a scrollbar to access others). For each room, a room menu
(see figure 12a) enables different actions to be performed: create a new
room; enter another room; delete the current room (only allowed if you
are room owner); erase the whiteboard; empty the room; find out the
room properties (i.e., who owns the room, etc.); set room permissions;
80
Chapter 5 Teamwave workplace
retrieve old versions of the room; save the current version of the room;
save the room as a template; and set the room as the default room when
logging in. The room owner can set permissions for the room (see figure
12b) such as who can enter the room, create tools in the room, delete
tools in the room, change the whiteboard, etc. In figure 12b the column
on the left indicates the permission type to be set and the right column
gives the choices of which group can be given that permission.
Figure 12
Room and permissions menus in TeamWave Workplace
As mentioned earlier, each room can be customised by the team to suit
their specific needs and tasks by using any of the 19 tools provided in
TW. The use of the tools is relatively intuitive just by seeing the name of
the tool (e.g., address book, database). The tools include: Address Book,
Brainstormer, Calendar, Chat, Concept Map, Database, Doorway, File
Holder, File Viewer, Image Whiteboard, Meeting Roster, Message Board,
Personalised Message, PostIt, ToDoList, URLRef, Vote, Web Browse,
and the on-line help. Figure 13 illustrates a doorway tool (e.g., doorways
to Help Room, Training Room, and Team01…Team11 rooms), a calendar
tool, an address book tool, a To Do List tool, a URL link tool, a file
holder tool (VisArt.rtf), and the chat area.
The tools can be used for navigation, production, communication, management and consulting. Navigation tools provide means for navigating
between rooms (doorway tool) or to information (URLRef tool). Production
tools (brainstormer, concept map) enable team members to collaboratively
share ideas, make group decisions, or build knowledge representations.
Communication tools provide a way of asynchronously communicating
information (file holder, message board, vote, Postit) or synchronously
81
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
communicating with one or more team member(s) (chat, page/personalised message). Management tools (address book, calendar, meeting
roster, To Do List) allow team members to coordinate their work. Finally,
consulting tools (database, file viewer, image whiteboard, web browser,
on-line help) provide access to shared information or help.
Figure 13
Classroom – An example of a TW room and its tools
Awareness
As described in section 2.5, Gutwin et al. (1995) present a framework of
awareness for collaborative learning which comprises four types of
awareness: social awareness, task awareness, concept awareness and
workspace awareness.
TW pays close attention to workspace awareness by providing a number
of features that maintain awareness of other team members, and of other
82
Chapter 5 Teamwave workplace
users who are on the same server18 . For example, as illustrated in figure
13, TW displays separate lists of users in the current room and a general
list of other users on the server. The current room users list contains either
the user’s name or, if desired, a small picture of the user. In figure 13 this
can be seen in the Now in Classroom list with a picture of Barbara and
Eirin’s name and how long Eirin has been idle. Moving the mouse pointer
over the picture reveals whether the user is active or how long they have
been idle and indicates the colour of that user’s telepointer on the
whiteboard. Clicking the mouse button over the name or picture gives
access to the user’s business card which contains more information about
the user including a phone and fax number, email address, URL homepage
address and (if entered when the user logged on) the current physical
location19 of the user and a phone number where they can be reached. It
is also through this clicking on the name or picture that one is given a
choice of sending email, accessing the user’s home web page or paging
the user (i.e., send a short personalized message that to get the user’s
attention). The displayed general list of other users on the server, gives
their name, the name of the room which they are in and the length of time
they have been idle. This list can be seen inn figure 13 in the Other
Connected Users list where there are 5 visible names and a scrollbar to
read the rest of the list (at the time the screen shot was taken there actually
were 12 users listed in this list). A double click on a name in the general list
gives the business card of the user. These simple mechanisms (displayed
lists of users) play a significant role is providing a general awareness of
who is around (the general list) and a more fine-grained awareness of
other team members’ actions (current room users list).
Another fine-grained sense of awareness of other team members’ actions
is provided through telepointer. A telepointer shows up as a coloured
dot, one for each team member, on the shared whiteboard and follows
each member’s movements. These enables team members to follow what
team-mates are looking at. When a tool is moved around on the shared
18
19
These users may be on different teams and have access to different areas of the
server. For example a class of students may be divided into various teams for a
particular activity and each team assigned their own private team room. In
addition, there maybe other rooms which all teams can access (e.g., a room
describing the activity, a resource room, a help room, a coffee room, etc.).
This feature of TW also facilitates access to each other enabling, for example, a
phone call to the person or a walk around the building to where the person is
located.
83
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
whiteboard, other team members see not only the result, but see who is
moving the tool. Thus users see not only each other and what they are
doing, but see immediate changes in the room’s artefacts.
A room overview radar is used to provide awareness of the location of all
room tools on the shared whiteboard. This miniature room overview of
the room is necessary to facilitate the use of a shared whiteboard that is
bigger than what will fit on a display (remember that not all users will
have a large 21 cm display and maybe working on a 12 cm display). A
miniature telepointer is also provided on the room overview radar, so as
team members move around the room, the radar tracks their actions
(Roseman & Greenberg, 1996).
Concept awareness supported in TW is directly related to the learners’
collaborative construction of knowledge. For example, concept awareness
of a file that has been placed on the server using the file holder tool is
provided by giving information about the size of the file, who put it on the
server, and the date and time of the last modification when the mouse
pointer is moved over the tool. Similarly, a history of each tool can be
retrieved through the tool’s menu (a menu for each tool is created when
the tool is created and is visible when that tool is activated) that tells the
name of the tool creator or modifier, the date, time and size of the information at creation or modification of the tool.
In addition to these automatic awareness features, team members can use
the provided tools to provide awareness. For example team members can
use PostIt notes to indicate work that they have done while the others have
not been around and to indicate when they have been present in a room.
5.2
User interface considerations
The TW interface seems to support what Fjuk and Øgrim (1997) suggest
as an action-oriented approach to computer support for collaborative
learning. All actions are connected to the different information objects
and the interface provides an intuitive indication of what operations the
provided tools support. The TW interface can be described according to
Appelet et al.’s (1998) characterisation of a system’s interface:
Visibility of the different types of information objects. TW’s use of
the shared whiteboard for placement of shared information artefacts provides
84
Chapter 5 Teamwave workplace
signs to team members that information is available for individual knowledge
construction and for collective growth. When an interactional aspect of an
action requires an operation, TW provides a number of mediational tools
that are both immediately visible (chat and lists over others in the room and
in other rooms, white board, help menu) and easily accessible through the
tool menu. Once a tool artefact is created, a menu corresponding to that
tool is created and is visible whenever the tool is activated (by clicking on the
tool). The list of available rooms (including who is in the room) provides
structural information about the teams working area.
Visibility of what actions can be performed. Awareness of what
operations can be performed by what tools is fairly intuitive in TW. Each
created tool artefact’s menu gives a number of operations that can be
performed on the tool artefact including: minimise the tool; delete the
tool; copy it to another room; copy it to the server; find out its properties;
get a version history; and save a version of the tool. In addition, the
available operations are either explicit (fill in the slots in a database, or
enter an item in the ToDoList) or are found by visiting the tool artefacts
menu or by clicking the right mouse button over the tool artefact (e.g.,
visiting a doorway artefact menu or clicking the right mouse button over
a doorway artefact gives the option of entering the room to which the
doorway leads, or changing the room to which the doorway leads).
Layout of the interface. TW allows users to customise the user interface
to reflect individual and team needs. Upon creation of a new room, the
user is given an empty whiteboard where various tool artefacts (e.g., information or communication artefacts) can be placed. Each information object
is given a name and several operations can be performed on the object
(recall descriptions above). TW’s interface appears the same to both novice
and experienced users. Since the room begins with an empty whiteboard,
it is up to each individual or team to maintain and control the “clutter” of
the room (i.e., organise the room for their own individual or team’s use). It
is not possible, however, to customise the standard tool menus.
5.3
Production, Communication and Distribution
In Fjuk et al. (1999) it was reported how TW supports production/
knowledge construction, communication and distribution/division of labour. This section summarises these results.
85
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Production/Knowledge construction
TW supports, in a limited way, searching for information. The construction
of a personal domain for information and knowledge is facilitated by the
possibility to create one’s own personal room, which can be customised
as desired. Limited support is provided for articulating meaning (thought
beliefs, knowledge, experiences, and skills) into various information objects
(plain text, hypertext, pictures, drawings, schemes, informal notes, etc.).
Producing information objects for distributing them in the collaborative
community can be either carried out in the individual’s own working environment (e.g., using favourite word processor or spread sheet) and
then sharing them through TW, or using some of the tools to provide
information objects for team-mates to examine. TW also provides support for distributing and co-ordinating meanings to peers, enables
individuals and teams to reflect upon and elaborate information objects,
and supports the planning of an individual’s own and collective progress.
Tools are provided to support the identification of one’s own role and
tasks with respect to the collaborative community’s division of labour. No
explicit support is provided for (re)evaluating ones own knowledge and
interpretations, although such support can be designed by the individual,
a team or a facilitator20 . Table 5 lists the TW tools that support the
operationalisation of these interactional aspects of actions.
20
86
For example, in our use in a course at the University of Bergen we created a
self-evaluation form on the web which we linked to in TW, that helped the
student evaluate their confidence in using TW.
Chapter 5 Teamwave workplace
Table 5 TW tools facilitating production/knowledge construction (Fjuk
et al. 1999)
Operational aspect of actions
TW tool/feature
Digital libraries and corresponding
functionality for searching for information
Database, URL links, Web Browse, File
Viewer
Mechanisms for constructing personal
domains of selected information objects.
Rooms
Mechanisms for articulating thoughts
White board, concept map, file holder
Mechanisms for simple downloading and
uploading information objects.
File holder, URL holder
Mechanisms for simple attachment of
information objects in social interactions
Mechanisms for making notes
Postit,
Mechanisms for personal planning
Calendar, ToDoList,
Self-evaluation tools
Postit notes (with comments from peers),
URL link (to self-evaluation questionnaire)
Communication
TW supports the distribution and co-ordination of information among
peers. There are several ways to comment on information objects provided
by peers and to discuss relevant topics connected to the problem/subject
under consideration. Mechanisms for making common decisions are
provided. Limited support is provided for negotiating meaning, coproduction/co-authoring of information objects and, while getting an
overview of the peers’ action and progress is well supported. TW is focused
on providing means of creating a common presentation area and enable
users to create content annotations for each other.
Table 6 lists the TW tools that support the operationalisation of these
interactional aspects of actions.
87
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 6 How TW facilitates communication (Fjuk et al. 1999)
Operational aspect of actions
TW tool/feature
Mechanisms that retrieve information
about social interactions
Telepointers, Immediate update of
information objects when changed, PostIt
(to record information about who was here
and when)
Mechanisms that retrieve lists of all actors
in the social communities and other actors
in the system
Address Book
Mechanisms that inform about which
actors that are currently on-line.
General user list, Room user list
Mechanisms for interacting both
synchronous (e.g., chat) and asynchronous.
Chat, Page/Personal Message,
Brainstorming, Message Board
Mechanisms that retrieve a whole dialogue, Chat (saving to own working area and
possibly containing contributions before and sharing with team members), Message
after the one specified by the actor.
Board, Brainstormer
Mechanisms that search across the dialogues.
Mechanisms for categorising dialogues
item according to keywords, chosen by the actor.
Mechanisms for producing joint information
objects
File holder, Concept Map, White Board
Mechanisms for voting among alternative
positions and meanings.
Vote
Mechanisms for providing decision support.
Vote, Brainstormer
Distribution/Division of Labour
TW provides several mechanisms to support the organisation of a team/
project enabling the articulation of responsibility and mutual commitments
(who is doing what when) and articulation the use of time (for meetings,
deadlines, etc.). Making common plans is supported in a limited sense as
is providing awareness of how a particular activity fits into the individuals’
knowledge, doings and progress.
Finally, support is provided with respect to up-to-the minute knowledge
about peers’ interactions with the collaborative environment.
Table 7 lists the TW tools that support the operationalisation of these
interactional aspects of actions.
88
Chapter 5 Teamwave workplace
Table 7 How TW facilitates distribution/division of labour (Fjuk et al.
1999)
Operational aspect of actions
TW tool/feature
Mechanisms for retrieving an overview of the ToDoList
individual’s status in relation to her
commitment in the social community.
Mechanisms for supporting project
management.
ToDoList, Vote, Address Book, Meeting
Roster
Mechanisms for using a joint calendar facility. Calendar
5.4
Technical requirements
TeamWave Workplace maintains a web site at http://www.teamwave.com
where interested users can download a free Workplace client and get
information about connecting to a demonstration server before committing
to purchase. To purchase TeamWave Workplace, you buy a workplace
license (valid for 1 year of updates) according to the number of users you
wish to support and according to whether you buy a regular or educational
license. In March 1999, a single regular license cost $78 US and a single
educational license cost $40 US while 50 regular licenses cost $1999 US
and 50 educational licenses cost $899 US. Purchase before March 15th
and there was a 10% discount (there seem to be several such promotions
throughout the year). It is also possible to get a demonstration license that
will allow you to test TeamWave Workplace for a period of 3 weeks.
Clients and servers are available for the following platforms: Windows 95/
98/NT, MacIntosh, Linux, Solaris (aka SunOS 5.x), SunOS (4.1.4 aka Solaris
1.x), SGI (IRIX 6.2), and AIX. This means that each user can decide for
themselves which client version they will download. It is also possible to launch
Workplace from a web page. The minimum ram required for the server is 16
Mbytes, but 32 Mbytes is recommended. The required disk space depends on
the activity the server sees. Every file or graphic that is put to a workplace
server takes space and is not automatically removed when a room is deleted
or its contents is deleted. This is due to the versioning feature which allows the
recall of prior versions of a room’s contents. A client requires only enough disk
space to run which is about 4 Mbytes. As for the server, 16Mbytes of ram is
enough but 32Mbytes is better. According to Roseman & Greenberg (1996),
TW has been designed to not require a very high bandwidth for network
89
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
connections. This means that users connecting over 14.4 modem connections
or long distance Internet connection find it reasonable to use TW.
5.5
Final remarks on TeamWave Workplace
There are several reasons why TW was chosen for use in the DoCTA
scenarios IDEELS and VisArt. First, the focus in TW is on the support of
collaboration itself rather than on instruction and scaffolding. The emphasis
is first and foremost on the collectively-oriented aspects of collaboration
through the facilitation of the creation of a shared networked place and
awareness, and secondly on individual aspects (e.g., by enabling the
creation of a room for individual use when individual tools can be placed).
Furthermore, a major strength of TW is its support for both pre-planned
and spontaneous interaction among team members.
The second major strength of TW is that it provides an integrated set of
general tools that can be placed in customised rooms for team or individual use. The individual continues to work independently on their own
system using the tools they are used to and uses TW to share created
information with team-mates.
A final aspect that has not been discussed to this point is the tailorability
and customisation of TW. TW is tailorable by the administrator/facilitator
and developers, and customisable by the individual/team and by developers.
The administrator can adapt TW for local use by creating teams of individual users and giving the permission to access particular rooms on the
server. A facilitator (e.g., a course instructor) can tailor TW to have a set of
rooms for the students (e.g., a help room, a training room, a classroom,
and team rooms, activity rooms, etc.). As discussed earlier, the individual
user or team can customise individual or team rooms as desired according
to need and tasks. Finally, as TW has emerged from a University research
group, the developers have had a very open attitude toward enabling
developers to tailor TW to their own use. Purchase of TW licenses also
provides access to a Software Development Kit (SDK) that enables one to
extend the functionality of TW by developing their own tools (see http://
www.teamwave.com/newtools.html for hints on how to do this). In VisArt
we tried to tailor further development of the application itself and ran into
problems. We received help from the TW developers, as there were certain
development aspects they needed to do themselves. This can be seen as a
drawback with regard to tailorability.
90
Chapter 6 Ideels
Part II
The Scenarios
IDEELS
VisArt
91
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
92
Chapter 6 Ideels
6 Ideels
Project IDEELS (Intercultural Dynamics in European Education though
on-line Simulation) brings together a diverse group of educators and
researchers from five tertiary institutions in four European countries who
share a common interest in simulation and games (see Appendix E for
further information). IDEELS is an EU Socrates curriculum development
project with partners at the University of Bremen, Germany (coordinators), the Polytechnic University of Valencia, Spain, the University
of Nice, France, and Nord-Trøndelag College (HiNT) and University of
Bergen (UiB), Norway (see figure 14). The goals of IDEELS include adding
impetus to the curriculum development trend towards content & processbased learning and to enhance European competitiveness by providing
students with opportunities to learn essential cross-cultural, linguistic and
negotiating skills.
A generic simulation game is used to complement existing curricula in a
wide range of areas including language learning, negotiation, policy studies,
political science, environmental issues, cross-culture communication, law,
education, and computer science. In IDEELS simulations, students act as
high-level negotiators, consultants, and journalists in a fictional world,
working to resolve real problems of importance to the European Community – problems that can only be solved through co-operation at the
international level.
93
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 14
Screenshot from the IDEELS’ homepage
Teams from the various partner institutions take on different roles in a
simulation conducted on two levels: deliberations within a team and
negotiations between or among teams. Thus, it can be said that the
simulation scenarios are designed to require (inter)dependence among
both team members and between teams (Wasson & Mørch, 1999). Teams
are given a common mission where the goal is to produce, agree upon
and ratify a jointly written document (e.g., come to a consensus and sign
either a policy or treaty, or write a set of recommendations). In the November 1998 simulation, teams from the various partner institutions
represented a country, a technical consulting company or a newspaper.
The overall goal or the common mission these teams had was to produce,
agree upon and ratify a jointly written set of recommendations for a
design plan for the educational system for Eutropolis, the New Eutropian
Capital. The first task each team had to do was to prepare two documents:
an Internal Briefing Document, intended to guide the team’s actions
94
Chapter 6 Ideels
through the negotiation, and; a Position Paper for “public consumption”
stating the team’s initial negotiation position. Other specific tasks that
the teams had to engage in included a number of real-time online
teleconferences, where the different parties should discuss a pre-selected
topic. The teams were also encouraged to engage in an ongoing dialogue
by sending each other questions, inquiries and request clarification of
different statements.
To communicate their policies and conduct a dialogue with the other
teams, all participants had access through the World Wide Web to OPUSi
a computer-based communication system developed at the Department
of Computer Science, University of Bremen for use in IDEELS simulations.
OPUSi enables participants to send each other written messages (within
a team and between teams), and to participate in real-time online
teleconferences.
Norwegian Participation
Two students from Nord-Trøndelag College participated in the IDEELS
simulation during November 1998 were students in the Sociology
programme in the Department of Social Sciences. The students
participated as a team representing a country, Bardland. An attempt to
recruit about 8 or 10 students failed, primarily because the participation
did not count towards meeting a course requirement. Since these were
volunteer participants, the students did not have any special reading
assignments with respect to role-playing.
Two teams of students from the University of Bergen participated in the
November 1998 scenario as part of a graduate course in pedagogical
information science. Six students volunteered to participate in the IDEELS
scenario and write a term report based on the experience. The students
were divided into two teams of 3 students and each team was assigned a
role by the IDEELS facilitator at the University of Bremen. One team
played a country, Highland, and the second team were journalists. As
the participating Bergen students were geographically spread over Norway, it was decided that OPUSi did not provide enough support for intrateam collaboration, so it was supplemented with both team email
addresses and a Teamwave Workplace. In addition, the students had
access to their own PC environment with whichever word processor and
Internet browser they preferred (see figure 15).
95
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 15
Supporting tools in the scenario
The collaborative telelearning scenario that is the focus of this study is
not only limited to include the actual IDEELS scenario. As mentioned
previously, the students’ participation in the IDEELS scenario was a part
of an assignment in a research seminar in pedagogical information science
(see Appendix F for the assignment and the course description). In this
assignment they were instructed to evaluate the scenario from one of the
perspectives covered in the class viz.: psychometrics, distributed cognition,
activity theory, or discourse analysis. They were supposed to write a
project description for a study, including a research question, the type of
data to be collected, what methods to use in this data collection and
account for the perspective they had chosen. The result of this assignment
should be a written report that was to be handed in for evaluation and be
given a grade.
The collaborative telelearning scenario is thus used to denote the team’s
entire collaboration effort, starting with the participation in IDEELS and,
subsequently, collaborating to finish the assignment.
OPUSi
OPUSi is a web-based communication system that is available from a
server at the University of Bremen, Germany, and can be accessed
through a common web-browser, such as Microsoft Internet Explorer or
Netscape communicator. The OPUSi system is specially designed to support the different activities in the IDEELS scenarios.
The user interface of OPUSi is divided into a number of “frames”, where
the top frame contains a menu showing the various user features available
96
Chapter 6 Ideels
in the system (see figure 16). This navigation bar has a list of options that
includes: Internal memos; Message centre; Papers; Library; Conference;
and Logout.
The “internal memos” option can be used to both send and receive
asynchronous messages within a defined user group (a team). The
“Message centre”, on the other hand, supports inter-team asynchronous
communication. As you can see in figure 16, the “Message centre”
contains a “Folder” for each team, and each of these “Folders” (“EuGlobe”
in the figure) holds the messages submitted by the actual team. All
members of a particular team have the same “view” within the system
and thereby have access to the same areas and documents. This means
that, for example, in the “Message centre” a specific group can read the
messages that are sent to everyone and the messages that are addressed
specifically to this group. The “Library” area contains resources and
background information (e.g. about the Maastricht Treaty) about certain
topics that is addressed and discussed in the scenario. The “Conference”
choice gives access to and support online real-time teleconferences. A
conference is usually initiated by a team of facilitators in the scenario and
has a specific topic, but all teams have the possibility to start a conference.
After a conference is ended the written discourse is still available for
review. The “Papers” option is dedicated to support collaborative (not
simultaneous) writing and editing of the papers each team is supposed to
publish, for example, in the “message centre”. In addition, the system
contains very basic word processing facilities within the different menu
options that enable the users to compose written messages.
97
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 16
Screenshot from OPUSi
In other words, OPUSi provides its users with the possibility to engage in
synchronous and asynchronous communication, collaborative writing and
helps them find relevant resources.
6.1
Evaluation of intra-group collaboration
FRODE GUIBYE
This section describes an evaluation (Guribye, 1999, Guribye & Wasson,
submitted) of the intra-group collaboration of one of the teams from UiB
that participated in the IDEELS scenario and subsequently wrote an
assignment about their own participation in this collaborative telelearning
scenario.
This team of students had been assigned the role of journalists who were
responsible for publishing a newspaper or magazine. As journalists, they,
98
Chapter 6 Ideels
in addition to producing a periodical edition of their publication (at least
twice a week), had to regularly interview the other teams and provide
reports from the different events and developments in the scenario. In
the teleconferences they had a more passive role and were only supposed
to join these conferences as observers without any active interventions in
the actual discourse. The first thing the team decided was that they wanted
to produce an electronic newspaper, called “NewWave” and publish it
on the Internet.
First the focus of the research study is given, then an overall description
of the teams’ collaboration process is presented, and finally some
concluding remarks and possible implications of the particular study is
given. In this way the goal with this part of the report is to give a “thick
description” (Geertz, 1973) of one of the teams that were actively
engaged in the IDEELS scenario, and from this description try to give
some concluding remarks concerning what aspects one should be aware
of when designing a collaborative telelearning scenario.
The Research Study
The initial empirical investigations in this study, were centred around one
tentative research question:
How do students organise their work?
The design of the scenario provided a setting for the students to carry
out their tasks. Prior to the actual study, relevant elements of this context
were considered to consist of: actor characteristics, the kind of artefacts
provided, the kind of artefacts they were to design and the design of the
learning activity. During the study the focus was narrowed down and
directed towards some aspects of the way the students organised their
work. It was possible to identify more precise questions, such as: “how do
the students collaboratively arrive at a particular division of labour?”
“what modes of communication do the students choose?” and “what tools
do they use to mediate their communication?”.
99
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Data Collection
The data collection techniques that were applied in the actual data gathering were mainly selected from the repertoire of techniques available
within ethnography (Hammersley & Atkinson, 1983). Some of these
techniques had to be especially adapted (as always) to the specific and a
bit unusual context and environment. The most extraordinary dimension
of this was that large portions of the group’s communication were computer-mediated. This led to a question about how to do computer-mediated
“field” observations. The answer is given partly by trying to immerse
oneself in the, so called, virtual environment and partly in improvising
and being creative when searching for possible sources of information.
Table 8 lists the techniques used in this study.
Table 8 Data gathering techniques
Data gathering techniques
Remarks
“Field” observations
Informal, limited interaction, including the
bservations in the virtual environment
Collecting email
Automatically received from the team’s
“mailing list”
Interviews
In-depth, semi-structured, open-ended,
individual
Data logs, chat logs
From Teamwave, chat logs saved by one of
the students and emailed to the
researcher
Informal interview / email interview
With a second group
Described chronologically, the empirical study began with brief informal
conversations with the group members and occasional drop-in visits in
the data-lab where the one student situated in Bergen was working. These
visits can be categorised as “field” observations with limited interaction
from the researcher. This means that the interventions with the members
are mainly confined to seeking clarification and the meaning of ongoing
events.
During the entire IDEELS scenario, the members of the group
communicated frequently using email. All these emails were continuously
collected and read. Being able to follow the communication on email as
it arrived played an important role in the evolving “field observations”. In
100
Chapter 6 Ideels
this way it was possible to get a picture of the progress and the different
activities they were involved with at various stages in their collaboration
effort. The preliminary and ongoing textual analysis of the information
provided in the emails played an essential role in picking out suitable
data gathering techniques for further evaluation and in developing a
more particular focus for the study. At the same time their emails served
as documentation of the communication. Reading and rereading these
was a very valuable source when doing the post evaluation trying to
reconstruct a picture of the already past events and activities.
OPUSi also mediated communication during the scenario. This tool was
used mainly to publish produced documents. It soon became clear that
OPUSi played an insignificant role in the intra-group collaboration, at
least as a mean for active communication. It was rather a reference spot
the students only observed and on which they commented. At the very
beginning of the scenario the application was occasionally browsed to
check for possible intra-group activities. Most of the documents submitted
here were directed towards inter-group communication and these
documents were usually interchanged via email prior to their submission.
Closer to the end of the collaboration process, TW was used quite
frequently. Thus, observing and following the students’ activities in this
environment came to the fore. Although many of the different tools within
TW were used, the chat-logs that were collected from six real-time
meetings seem to be most important for evaluation purposes.
Interviews were used to follow up the data gathering that went on during
the group’s work. These interviews were carried out after the scenario
was ended, the assignment was due, and to cut the students some slack
after their oral exam. So, approximately a week after the collaboration
effort, an interview with each of them was arranged. To derive a structure
for the interviews, decide what topics that should be addressed, and to
design the main questions AT was used as a conceptual resource. The
resultant interview guide can be found in Appendix G. This means that
when preparing the interviews by analysing the already gathered data,
concepts and ideas from the theoretical framework were actively used to
understand what had been observed and to develop new ideas about the
way that the students organised their work. Also, the ongoing, preliminary
analysis served as a basis for selecting a certain focus in the interviews.
The interviews had two functions: to clarify what activities each student
101
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
had been engaged in, and to get insight into their view on why the
different activities had taken place and why they chose to do them in this
particular way. This whole process was very useful for identifying essential
elements in their colloquial activity.
The three individual, in-depth interviews can be labelled as semi-structured
and open-ended. Four main topics or categories were outlined, and some
questions about each topic were designed. This structure was used for all
the interviews, but many “follow-up-questions” arose spontaneously from
the situation or from the specific answer to a question.
This description of how the data gathering was performed in this empirical
study gives an actual example of what challenges that face researchers
studying actors collaborating (partly) over the Internet, and how these
challenges might be met.
Data Analysis
The analytical strategy used in this study included textual analysis (of
emails and chat-logs), an analysis of events and process, and a contextual
analysis where activity theory was used as a conceptual resource. In
addition, some effort was put into analysing the use of the different artefacts. Overall, this strategy aims at gaining a holistic understanding of the
collaboration process. Details of the analysis can be found in Guribye
(1999).
The group’s collaboration process can be divided into two main phases
(see figure 17), which have basic differences in terms of the tasks in
which they were engaged, the objectives they had, and the collaboration
patterns that emerged. The first phase is referring to the period when
they were participating in the IDEELS scenario (28/10 – 13/11). The
following weeks (14/11 – 30/11) are denoted as the second phase. In
this period they were engaged in carrying out tasks related to the
assignment. The first few days of this second period can be said to be an
‘intermezzo’, where there was practically no collaborative activity at all
could perhaps be described as “an anticlimactic” reaction after the scenario had ended. Overall, the two phases constitute a useful distinction
for analytic purposes.
102
Chapter 6 Ideels
Figure 17
The two phases in the collaboration process
Characteristic to the first phase is the fact that the students were quite
confused about both their intended role in the entire scenario, and the
lack of a (for them) meaningful objective and significant motivation for
their participation. All the same, the students ambitiously began to carry
out the tasks and collaborated eagerly in the initial stage
Their first introduction to the scenario was given very briefly at one of the
lectures in the course they were taking. Beyond this, they only got the
instructions and descriptions available from the web page (http://
www.zait.uni-bremen.de/wwwgast/fzhb/ideels/public_html/links.html) for
the Eutropolis scenario. The first intra-group communication in relation
103
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
to the scenario was in a spontaneous face-to-face meeting, where they
discussed their role in the scenario and also decided that they wanted to
make a web-based newspaper. After going back to their respective
hometowns, they used email to reignite the communication.
The first task in the scenario was to publish an “internal briefing document”
and a “positioning paper”. One of the group members wrote drafts of
these, shared them as email attachments and then the other two members
read and commented on these. This became the way they generally
collaborated when producing documents for the scenario. In the “internal briefing document” they explicitly assigned internal roles for each of
the group members with respect to their status as producing a newspaper. The roles were labelled as web-editor, journalist and editor with the
responsibilities for designing and updating the WebPages, designing
questions and ”interviewing” the other teams in the first phase, and writing
and editing articles for the newspaper, respectively. When they divided
the roles between themselves, they did this according to their different
skills and experiences. For example, one had done some web design
and volunteered as web-editor. These roles and their responsibilities were
not rigorously applied to guide their division of labour and collaboration.
As the scenario was running, new roles evolved and emerged. It was only
the work with the Web Pages that was handled by one student, but this
same student also had other roles to fulfil. At a later stage in this phase,
they continued to publish some articles and questioned some of the other
teams. The collaboration patterns varied as new tasks had to be done.
When they were writing articles, one of the students took the initiative.
But this student didn’t feel quite confident about his English proficiency,
so he wrote them in Norwegian and sent them to the others so they could
translate them to English, since the two others had more experience
with writing this foreign language. At the same time, the student that
wrote the drafts in Norwegian might be characterised as the one who
had the best overview over both the scenario and the topics discussed
there. So we see clearly that they choose a division of labour where they
all get to do what they do best, and in this way pooling their abilities and
resources in a (seemingly) optimal way.
Another task they were obliged to in the scenario was to follow the
conferences. In relation to this task another division of labour was chosen.
As all of the group members were quite busy with other assignments,
work etc., all of them didn’t have time to attend all of the conferences.
Instead, usually, only one student followed each conference, saved it as a
104
Chapter 6 Ideels
file and sent this to the others by email. One rather significant event
early in the first phase was when one of the students tried to communicate
to the other group members through the internal messages in OPUSi.
He submitted his message, but had to wait for approximately half an
hour before it was displayed in the system. Experiencing this kind of
delay in the system, was a contributing factor for not using the internal
communication facilities in OPUSi, and for choosing email as the main
communication medium, at least in this phase. Their use of email had a
peak in the very beginning of phase one. The same day as they published
their first “articles” in the scenario, they sent over twenty emails to each
other. After this the activity on email stabilised at around three to four
email per day. They also had similar experiences with TW. During the
first days, two of the students tried to use this tool, but one of them were
frequently “kicked-off” due to technical problems. They also had problems
with getting into the “room” created for the their group, and after a
couple of days they also gave up on using this tool. Actually, the logs
from TW indicate that one of the students did not log on to TW until the
last week of phase two. Overall, after having started with a very high
activity level, some confusion and with a considerable amount of
communication, by the last one and a half weeks of the scenario they
had reached a more stable division of labour and less communication
took place.
In the second phase distinctively different collaboration patterns emerged
- they used the provided tools differently, had a clearer objective, and
had more discussions on specific topics and tasks. This phase also began
with a face-to-face meeting in Bergen. At this meeting they tried to get
an overview of the task - what had to be done? In one of the interviews
this meeting was characterised as very efficient, because they had ideas
and suggestions that they could easily exchange and elaborate upon.
They also divided the assignment into subtasks and made a preliminary
division of the main responsibility for these subtasks between them. This
seemed (as perceived by them) to be a fruitful way to start the collaboration
with respect to the assignment.
After this they went each to his/hers town, and did some work on their
own. They then exchanged the produced documents (drafts) with each
other (first using email) and made comments on this work in the email
body. The last week, they also used TW extensively, having (at least) six
long chat-sessions and also used the facilities for asynchronous
communication provided in the software. During this last period the
105
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
collaboration was really intensified, and they needed a medium for
synchronous communication, and had to give the work with the assignment
a high priority (as we so often see when students have an assignment
due). Since they began using TW the number of emails dropped
significantly in relation to some of the peaks we saw in phase one, and
they “only” sent a couple of emails each day. Another reason that made
it possible for them to collaborate through TW was that the Teamwave
server was moved to another server thereby providing a more stable,
reliable and accessible service for them. Together with email they used
TW to exchange documents. By leaving the documents on the whiteboard
they could access each other’s files whenever they wanted. The coordination and version control regarding these files was to a large extent
managed by putting up “post-it notes”. They also used the highlight and
“track changes” tools in Word to comment on each other’s work. One of
them downloaded a Word document one of the others had authored and
highlighted or made a strikethrough (which means it should be deleted)
on certain sections and made editorial comments to these. This way they
worked closely together producing the documents, and had a subtle and
dynamic division of labour throughout the last part of the second phase.
The objective in this phase was finishing the assignment, and the outcome of the transformation process is the written document.
Analysing the context using the activity system
To really understand what was going on and how the students organised
their work, considerable effort had to be put into uncovering and describing
different aspects of the context. In doing this the activity theory framework
played an important role. It provides concepts and ideas that make it
easier to capture and analyse the context.
Figure 18 represents the collaboration process as an activity system
employing the general model of activity system proposed by Engeström
(1987). In this specific activity system the main components are the individual students as the subjects, the student group as the community, and
the object/objective is finishing the assignment. The activity was mediated
by a number of artefacts, including language (written and oral), their
email systems, TW and all its tools, OPUSi, and the WWW (the browsers
and the sites they visited and created).
106
Chapter 6 Ideels
Figure 18
The collaboration process as an activity system
The relation between the community and the subject was mediated by a
set of rules (both implicit and explicit) of interaction. These rules are
mostly present in the different cultures permeating the social setting in
which the activity took place. Certain rules and norms are more or less a
part of the, mostly, tacit body of knowledge that constitute the different
cultures, for example a university culture, the student culture or the
discipline culture. Interaction on the Internet can also be said to be subject
to certain rules and norms, and as all of the students had some experience
with this medium and its culture, this can be established as a contributing
factor in mediating the relation between community and subject. The
widely used and well-known Internet “genre”, including symbols such as
“smiley faces”, was to a large extent present in the intra-group
communication. This, actually, was somehow problematic at the
conferences in the IDEELS scenario. There the participants were supposed
to apply a rather formal “tone”, and when the students from this team
used (as they usually do when chatting on the Internet) “genre specific
signs”, the other teams didn’t understand what this was supposed to
mean (many of the other teams had less experience with communication
at the Internet), and it was interpreted as “inappropriate”. In the intra107
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
group communication, however, everybody was familiar with these
symbols, and they were used frequently in both emails and chats. At
large, all these norms and rules are not static, but are continuously being
constituted and dynamically evolve as they are used in new situations and
settings.
The relation between the community and object is, on the other hand,
mediated through a division of labour and the different roles and
responsibilities the students have towards the object. This division of labour is also changing and evolving throughout the collaboration process
and is closely related to the way the students divide the assignment into
tasks and subtasks.
Another distinction made by activity theory is that of the hierarchical
structure of an activity (Leontiev, 1978). The longer-term formation of
an activity where its object is transformed into an outcome, consists also
of shorter-term processes, namely actions and operations (Kuutti, 1996).
Analysing this structure in this specific context may contribute to the
understanding of certain events and happenings in the collaboration
process. The activity, as a whole, can be characterised as “finishing the
assignment”, and this is also the objective or motive for the activity (see
figure 19). Again, this activity consists of a number of actions directed
toward a goal. Examples of actions in this activity is “writing an email to
the other group members”, “participating on a conference”, “having an
online discussion in TW”, “publishing an article in the scenario”, etc. It is
important to note that the actions can be both individual and collaborative.
At the lowest level, all these action are realised as chains of operations.
As an example, the action “writing an email” consists of operations such
as “typing”, “selecting appropriate words”.
This hierarchical model of an activity reveals some problems that occurred
during the collaboration process. Especially in the first phase, the motive
or the objective was a little diffuse for the students. This means that all
the actions were not directed towards the same overall object and motive.
This is perhaps due to the fact that the students did not perceive a clear
purpose (other than “just participating to have something to write an
assignment on later”) when participating in the scenario. So, the only
motivation left then is the particular goals of each of the actions. In this
sense one could say that the actions, in lack of an overall object, have
shifted upwards to become activities. At the operation level, some
108
Chapter 6 Ideels
Figure 19
The hierarchical structure of an activity
technical obstacles (conditions) appeared, such as having to wait half an
hour for the internal message to be displayed in the OPUSi system. Then
what should have been operations required a fair amount of attention,
and were turned into actions directed towards conscious goals. When the
mediational tool (e.g. OPUSi) contains hindrances and requires much
attention from the users, it is easy to understand that they choose other
more familiar, reliable and “internalised” artefacts, such as email.
Highlighting these elements of the context makes it possible to better
understand the way the students organised their work and is at the same
time a frame for comprehending the different actions and events that
took place during the collaboration process.
Conclusions
Through this naturalistic study, significant aspects of the collaboration
activity have been identified. When looking at this activity from the angle
chosen in this study (strongly emphasising the social setting), it is apparent that – when using technology as a medium for communication and
collaboration – the social interaction is to a certain degree influenced by
the tools, but not fully determined by them. There are certain
characteristics of the social context and how the participants perceive
their tasks that are, at least, equally important to take into consideration
as the strictly technical aspects when looking at technology in use.
Of course, the technology in it self is a prerequisite for being able to
engage in computer mediated collaboration. If, at a very basic level, the
technology is not available or not reliable one can not make use of it as
109
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
a medium. Still, if all the technology (artefacts/tools) is provided and
functions properly, its potential users need, first, to have a sufficient
incitement and motivation to use it, and secondly, know how to use it,
and finally, to “trust” it and feel comfortable with using it. These three
considerations are not static and given, but are dynamically evolving as
part of complex social and psychological processes, and dependent on
the social context in which they occur.
When a set of artefacts are provided, forming a “virtual environment”
for the collaboration, the activities can be studied at another level.
Basically, one can say that (in this case) the students didn’t make use of
the different tools if they did not really need to use them. This is not one
of the most interesting and surprising findings in this study, but is in a
certain sense a basis on which the other findings rely. When looking for
collaboration patterns and work organisation in this context, it is important to take into consideration that they organised their work and that
the collaboration patterns emerged in a process mediated by the artefacts they used.
For example, when they, in the beginning of the first phase, came across
some technical and practical problems with TW, email took the role as
the main communication medium. As we all know, email is a tool that
only supports asynchronous communication, and this turned out to be
rather problematic in the sense that they really needed a tool for
synchronous communication, such as the chat-tool in TW, to support their
collaboration at this stage. As a consequence, the quality of their
communication suffered, and it was obvious that not all of them were
able to follow every aspect of this discourse when they tried to emulate
synchronous communication through email. Apparently, this example
illustrates the importance of having a good blend of tools supporting
both synchronous and asynchronous communication available in the virtual
environment. At the same time, acknowledging that they somehow did
solve the problems and tasks at hand, it suggests that the particular
medium or tool is influencing, but is not determining the course of the
collaboration activity.
Using concepts from activity theory, the mediating tools are only one
aspect of the entire activity that also appears at several levels (operations
– actions – activity). When everything at the operation level functioned
properly (technically speaking), the students could focus on their conscious
goals and thereby discuss the problems/tasks/topics at hand more
110
Chapter 6 Ideels
carefully, without having to pay much attention to the tools, as such. This
was the case in phase 2 in the scenario, when TW was used actively and
both synchronous and asynchronous communication was supported. Different collaboration patterns became visible and this clearly had a positive influence on the quality of the discourse. As always, this is not the
only contributing factor at play in this complex situation. The group had,
in this phase, better motivation and a clearer objective in their
collaborative activity, and this might have had a positive effect on the
discourse. This shows that through the lens of activity theory, it is not
sufficient to see only the technology or a tool in isolation. Rather, technology must be seen as an integral part of human activity and human activity
is always a complex and dynamic process that needs to be studied in
situ.
From an epistemological perspective, there are certain ways the findings of this study should be handled. In a qualitative case study, such as
this, the generalisations should be made from empirical data to theory
and concepts and not to a population of cases. In this way the concern
for the uniqueness of the particular setting and participants withstands.
Bearing this in mind, some concluding suggestions about what is important to take into consideration when designing collaborative telelearning
scenarios are listed. A crucial aspect is the importance of having a variety
of technologies (or one integrated technology with a variety of tools)
available to support both synchronous and asynchronous interaction in a
collaborative activity. In this way the social actors can swap easily between
different modes of communication depending on the particular need in
the specific situation. Other significant prerequisites include giving the
participants sufficient training in both use of the technology – ensuring
that they are comfortable with using it, and in how to integrate the technology in the learning activity. One last fundamental, and perhaps obvious,
suggestion is to provide the participants with clear objectives and goals
for the learning activity, and in this way arrange for proper motivation.
111
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
6.2
Changes in Attitude towards Simulation-based
Distributed Learning
KNUT EKKER
This section reports on an evaluation (http://wwwstaff.hint.no/~kne/
ideels/results) of changes in attitudes towards simulation-based distributed
learning in the IDEELS Scenario. All IDEELS participants, not just the
Norwegian students at HiNT and Bergen, could participate in the
evaluation.
This quantitative evaluation of IDEELS was administered with web-based
questionnaires. Pre- and post-simulation questionnaires (see Appendicies
H & I) were used and each student had a personal id code that was the
key to linking the pre- and post-simulation data sets.
Pre-questionnaire
The pre-simulation survey (http://wwwstaff.hint.no/~kne/ideels/
pre_sim_1998.htm) collected background data on the participants (e.g.,
age, year of university/college, native language, etc.) and asked questions
about their:
• attitudes and previous experience in using computer
• language related skills and attitudes, and
• experience with collaboration and team work
Forty-six students answered the pre-questionnaire.
Background information on participants
The background information of the participants shows that the teams
were quite different with respect to age, gender, year in university/college and whether they participated through a required class. Table 9
shows the distribution along these variables.
The students in Bergen and Steinkjer were generally older than the
students in Valencia and Bremen, and the majority of students were female in Bremen and male at the other sites. Students in Valencia and
Bergen/Steinkjer were more often in their 5th year of university/college
education whereas in Bremen the majority of students were in their 3rd
112
Chapter 6 Ideels
or 4th year of post-secondary education. The majority of the students in
Bremen, and all in Valencia, are participating through a language class.
Further the students in Bremen are participating as part of an elective
class, while most students in Valencia and Bergen/Steinkjer were
participating as part of a required class.
Table 9 Distribution of background information by Site
6.2.1.1.1 Variable
Age
Bremen
Total
(N)
40,0
40,0
20,0
100,0
(5)
69,7
21,2
9,1
100,0
(33)
12,5
37,5
50,0
100,0
(8)
56,5
26,1
17,4
100,0
(46)
Male
Female
40,0
60,0
75,8
24,2
62,5
37,5
69,6
30,4
1 or 2
rd
th
3 or 4
th
5 or later
20,0
80,0
3,1
25,0
71,9
25,0
12,5
62,5
8,9
28,9
62,2
Context in which
language class
you
technical class
are participating in interdisciplinary class
IDEELS
interdisciplinary class
60,0
20,0
20,0
100,0
42,9
57,1
80,0
2,2
6,7
11,1
75,0
25,0
71,7
28,3
Gender
Year in University
or
College
Participating through
a required class ?
20 – 22 years
23 – 25 years
26 years or older
Site
Valencia Bergen /
Steinkjer
st
nd
yes
no
100,0
81,8
18,2
Attitudes and previous experience in using computer
Table 10 shows that 75% of the students have a computer at home, and
that most students in Valencia or Bergen/Steinkjer have owned home
computers for four years or more. Fifty percent of the students or more
use the computer for an average of two hours or less per day.
113
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 10 Distribution of ‘computer at home’ and ‘computer use’ by Site.
Variable
Bremen
Computer at
home
I have owned
a computer for…
Average # of hours a
day using computer
Yes
No
Site
Valencia Bergen /
Steinkjer
Total
(N)
80,0
20,0
100,0
(5)
96,9
3,1
100,0
(32)
75,0
25,0
100,0
(8)
91,1
8,9
100,0
(45)
3 years or less
4 years or more
75,0
25,0
6,3
93,7
100,0
12,0
88,0
2 hrs or less
3 hrs or more
60,0
40,0
75,0
25,0
50,0
50,0
71,0
29,0
Experience with collaboration and teamwork
Table 11 shows the extent of experience with teamwork, role playing
and negotiating games distributed by Site. Most students have experience
with working in teams, while few students have experience with role
playing, and even fewer have participated in negotiating games.
Table 11 Distribution of experience with teamwork, role playing and
negotiating games by Site.
Variable
Bremen
Site
Valencia Bergen /
Steinkjer
Total
Experience with
working in teams
none / little
some / a lot
100,0
(N)
20,0
80,0
100,0
(5)
45,4
54,6
100,0
(32)
12,5
87,5
100,0
(8)
36,9
63,1
100,0
(45)
Experience with
role playing
None / little
Some / a lot
80,0
20,0
84,8
15,2
87,5
12,5
84,8
15,2
Experience with
negotiating games
None / little
Some / a lot
100,0
94,0
6,0
100,0
95,6
4,4
Post-questionnaire
The number of participants who completed the post-simulation survey
(http://wwwstaff.hint.no/~kne/ideels/post_sim_1998.htm) was 36,
compared with 46 participants in the pre-simulation survey. In Bergen/
114
Chapter 6 Ideels
Steinkjer the number of participants completing the post-simulation
questionnaire was reduced from 8 to 4 and in Bremen there were only 3
participants in the post-simulation survey. Thus, there will be some large
percentage-wise differences that may only reflect the fact that there were
only 3 or 4 participants in two of the sites.
Table 12 shows the evaluation of the simulation distributed by site. Four
individuals (13,8 %) in Valencia and two (50 %) in Bergen / Steinkjer
were neutral or negative in response to the item “I enjoyed this
experience”. Most of the participants felt this was a good learning
experience (over 90 % across the three sites), and most of the participants
felt this simulation fit well with the course they were taking (more than
73% across the three sites).
Table 12 Evaluation of the simulation by Site
Variable
I enjoyed this
experience
Bremen
agree
neutral or disagree
(N)
100,0
100,0
(3)
Site
Valencia Bergen /
Steinkjer
Total
86,2
13,8
100,0
(29)
50,0
50,0
100,0
(4)
83,3
16,7
100,0
(36)
This was a good
agree
learning experience neutral or disagree
100,0
96,6
3,4
50,0
50,0
91,7
8,3
The simulation fit well
agree
in the course I was
neutral or disagree
taking
100,0
75,9
24,1
50,0
50,0
73,6
26,4
Gender
In order to find out to what extent gender has an effect on the experiences
in the simulation, we present tables 13 – 16.
Table 13 shows that female participants were overall more positive to
the simulation experience (90 or 100% positive) compared to the males
(64 to 88% positive).
115
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 13 Evaluation of the simulation by Gender
Male
Gender
Female
Total
(N)
80,0
20,0
100,0
(25)
90,0
10,0
100,0
(10)
82,9
17,1
100,0
(35)
This was a good
agree
learning experience neutral or disagree
88,0
12,0
100,0
91,5
8,5
The simulation fit well in
agree
the course I was taking neutral or disagree
64,0
36,0
100,0
72,7
27,3
Variable
I enjoyed this
experience
agree
neutral or disagree
In table 14 we see that gender does not have any effect on the evaluation
of the simulation as a cross-cultural experience. In general, most
participants (60 %) felt that cultural differences were easy to identify, but
most participants (80 % or higher) did not feel that these cultural
differences made it difficult to communicate.
Table 14 Cross-cultural experience by Gender
Variable
Male
Gender
Female
Total
Cultural differences
agree
were easy to see
neutral or disagree
60,0
40,0
100,0
(25)
60,0
40,0
100,0
(10)
60,0
40,0
100,0
(35)
4,0
96,0
20,0
80,0
8,6
91,4
(N)
It was difficult to
agree
communicate
neutral or disagree
because of cultural
differences
Table 15 shows gender differences in the evaluation of the teamwork in
the simulation. All males feel that the teamwork functioned well with
respect to participation by all members, while a minority of the female
participants (22,2 to 30%) felt that not everyone participated in team
decisions.
116
Chapter 6 Ideels
Table 15 Experience with teamwork by Gender
Male
Gender
Female
Total
(N)
100,0
100,0
(25)
22,2
77,8
100,0
(9)
5,9
94,1
100,0
(34)
Everyone participated
agree
in team decisions
neutral or disagree
and product
N)
92,0
8,0
100,0
(25)
70,0
30,0
100,0
(10)
85,8
14,2
100,0
(35)
Variable
One or two
members took over
agree
neutral or disagree
Table 16 shows the overall evaluation by gender and shows that both
males and females felt that most, or all, of the goals or tasks were
accomplished in the simulation (88 vs 80%, respectively).
Table 16 Overall evaluation by Gender
Overall evaluation
We accomplished ..
.. some of our goals or tasks
.. most of our goals or tasks
.. all of our goals or tasks
(N)
Male
Gender
Female
Total
12,0
80,0
8,0
100,0
(25)
20,0
70,0
10,0
100,0
(10)
14,3
77,1
8,6
100,0
(35)
Knowledge of computers
Table 17 shows the effect of background knowledge of computers on the
overall experience of participating in the simulation. We see that a larger
percentage of the participants with an excellent knowledge of computers
prior to the simulation accomplished “all of our goals or tasks” (25%). Of
the participants with poor to good knowledge of computers, none
accomplished “all of our goals or tasks”.
117
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 17 Overall evaluation of the simulation by Knowledge of computers
Evaluation
Please rate your knowledge of computersOverall
poore-good very good
exelent
Total
I We accomplished ..
.. some of our goals or tasks
.. most of our goals or tasks
.. all of our goals or tasks
(N)
18,8
81,3
9,1
100,0
(16)
9,1
81,8
25,0
100,0
(11)
12,5
62,5
8,6
100,0
(8)
14,3
77,1
100,0
(35)
Attitude change
This section summarises the survey findings with respect to the research
question on attitude change.
Table 18 shows the attitude change with respect to the variable “A good
learning experience” by gender. The post evaluation score on this question
was subtracted from the pre evaluation score (a value of 1 indicated
“strongly agree” to the question “This was a good learning experience
(post)” and “I anticipate this will be a good learning experience (pre)”).
The table shows the majority did not show any attitude change (54.3 % indicate
“no change”). The table also shows differences by gender — males are more
likely to indicate an attitude change — about ¼ in either direction (“more
negative” or “more positive”) while female participants 70 % indicate no
change in attitude towards regarding simulations as a good learning experience.
Table 18 A good learning experience (attitude change) * Gender
Male
This will be/was a good learning
experience
(attitude change) Gender
Total
More negative 28.0%
No change 48.0%
More positive 24.0%
25
100%
Gender
Female
Total
10.0%
70.0%
20.0%
10
100%
22.9%
54.3%
22.9%
35
100%
Table 19 shows the attitude change with respect to the variable “The simulation
game will be (or was) difficult” by gender. The pre-evaluation score on this
118
Chapter 6 Ideels
question was subtracted from the post evaluation score (a value of 1 indicated
“strongly agree” to the question “I am concerned the simulation game will
be difficult (pre)” and “The simulation game was difficult (post)”.
The table shows that almost half (48,6) of the participants felt the
simulation was more difficult than expected — but when distributed by
gender we see that 60 % of the female participants expressed no or a
positive attitude change.
Table 19 The simulation game will be/was difficult (attitude change) *
Gender
Male
The simulation game will be/was
difficult (attitude change)
More negative 52.0%
No change 32.0%
More positive 16.0%
25
100%
Total
Gender
Female
Total
40.0%
30.0%
30.0%
10
100%
48.6%
31.4%
20.0%
35
100%
Table 20 shows the distribution of the attitude change variables by each
other. If you experienced the simulation as more positive than you
expected, you are more likely to express that this was a good learning
experience (28.6 % vs. 17.6 %). But just as many expressed that the
simulation game was more positive than expected, but found it to be a
more negative experience (28,6 %). It should be kept in mind that there
were only 7 individuals in this category (“More positive” with respect to
the difficulty of the simulation).
Table 20 A good learning experience * The simulation game will be/
was difficult
The simulation game will
be/was difficult (attitude change)
More
No
negative change positive
This will be/was a
learning experience
(attitude change)
Total
More negative
No change
More positive
23.5%
58.8%
17,6%
17
18.2%
54.5%
27.3%
11
100%
28.6%
42.9%
28.6%
7
100%
More
Total
22.9%
54.3%
22.9%
3
100%
119
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Other tables from the quantitative data analysis of Scenario I of Project
DoCTA / Project IDEELS may be found at this URL: http://
wwwstaff.hint.no/~kne/ideels/results/.
The discussion on attitude change in Project IDEELS shows that quite a
few of the participants had a more negative attitude towards the simulation
after completing the participation compared with the attitude they had
in the outset. This was particularly the case for male participants. Another
effect by gender is the fact that female participants were more likely to
experience frustration with respect to other team members “taking
control” over the team effort.
Two conclusions may be drawn from these facts. First, the software utilised
(OPUSi) may be improved in order to alleviate some of the frustration
experienced. Second, in preparing for future simulations it will be important to focus on integrating all members of the team in order to avoid
having one or two participants “taking control”.
120
Chapter 7 VisArt
7 VisArt
The VisArt scenario involved students taking courses at three educational
institutions, the University of Bergen (UiB), Nord-Trøndelag College (HiNT),
and Stord/Haugesund College (HSH). Teams comprised of 3 students, 1
student from each institute, collaborated to design a learning activity in
TW. There were no opportunities for the teams to meet face-to-face.
TeamWaveWorkplace (TW) was used as the main information and
communication technology. The VisArt activity took place during February
and March 1999 and provided an opportunity for the DoCTA project to
study an authentic collaborative telelearning activity. One week of training in using the TW tools and in collaboration proceeded three weeks of
design activity.
This section describes how we designed the VisArt scenario, discusses its
deployment and presents overviews of the evaluation studies on VisArt.
7.1
VisArt Design
VisArt was a somewhat special scenario. From its inception it was designed
to serve two purposes, each with different, but complementary goals. As
a course assignment the goals were to enable the students to:
• participate in an authentic collaborative telelearning activity
• evaluate their own participation from a theoretical perspective and,
the research goals were to:
• study / evaluate a collaborative telelearning activity from a sociocultural
perspective
• study / evaluate distributed collaboration artefacts
• find out how students experience it
• find out their own explanations and evaluation of the activity
121
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The design process
As indicated earlier, VisArt was to be part of three different courses at
Norwegian educational institutions. As the University of Bergen (UiB)
was leading the scenario, the VisArt activity, including the evaluation,
was designed by the UiB researchers and then sent to the other institutions
for reactions. This description describes aspects of the design process.
As Fjuk, Sorensen & Wasson (1999) argue, the design of telelearning
scenarios is a product of complex interactions between pedagogical,
technological and organisational perspectives. Each of these perspectives
place constraints on the design. Figure 20 illustrates the major constraints
that were placed on the design of the VisArt scenario. In addition, as
VisArt was part of a research project, evaluation concerns played a role in
the design.
Figure 20
Constraints on the VisArt design
The design of the VisArt activity took place from early November 1998
until mid-February 1999. This may seem like a long period, but one
aspect of telelearning activities is that a lot of preparation goes into
designing and preparing an activity. Tasks included in the VisArt design
phase were choice of design task, specification of the technological environment, coordination of dates for the activity, design of training, design
of help and assistance during the deployment, specification of the design
task, and design of evaluation instruments. In addition, coordination and
management of all these tasks needed to be carried out.
122
Chapter 7 VisArt
The design was initiated as a brainstorming activity at a DoCTA workshop
in early November 1998. In the project application it had already been
determined that:
• teams of students would collaboratively design a visual artefact, and
• a team of students would comprise one student from each of the
educational institutions
Previous work with the other scenarios and evaluations of collaboration
technologies, lead us to decide that:
• TeamWave Workplace (TW) would be the collaboration technology, and
• the students would use their own email system and web browser
During the workshop we decided that:
• the VisArt students would design a web page for teaching some topic
of choice
• two of the graduate students would implement a new tool, a web
palette, in TW to support this design
We decided that the first step was the implementation of web palette
and several meetings were held where progress was demonstrated.
Unfortunately, due to constraints on the type of tailoring allowed by the
TW developers, it proved impossible to implement the tool as we wanted.
At the time21 we were trying to add our own tool (i.e., November 1998)
TeamWave did not officially support the development of new tools. There
were guidelines for how to do so, but it was not easily done (e.g., the api/
documentation they provided was not sufficient enough, and the environment/framework also had “bugs” we could not work around). These
factors made it impossible to build our own tool and integrate it into TW.
Our only option was to change the design task. In December 1998 we
had another group meeting where we brainstormed about other possible
design topics and concluded that we would have the students design a
teaching/learning room in TeamWave Workplace. In retrospect, this was
a much better design task than the original we had chosen. Once the
design task had been decided upon, we could begin serious planning and
preparations of the VisArt design activity.
21
It is now, however, possible to develop your own tools through the inclusion of a
software development kit (SDK). The SDK includes complete documentation, as
well as the Tcl/Tk source code for all of the built-in TW tools.
123
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Modelling the VisArt Activity
In order to assist in management and coordination of all aspects of VisArt,
the activity was modelled22 in a knowledge modelling tool developed at
the LICEF Research Centre at Télé-université, Montréal called MOT
(modélisation par objets typés—modelling with object types). MOT (http:/
/206.167.88.162/anglais/real/mot.htm) is an object-oriented modelling tool for developing graphical knowledge models for learning
environments. The MOT editor enables easy creation of hierarchical graphs
that represent knowledge objects and the links between them making
the nature and structure of a learning system explicit. MOT is most often
used in conjunction with MISA (http://206.167.88.162/anglais/real/
misa.htm) a learning systems engineering method used to produce a
learning system which is characterised by a knowledge model, a
pedagogical model and a media model.
MOT provides the modeller with a set of knowledge types (concept
(rectangle), procedure or activity (oval), principle (rectangle with diamond
ends, or fact (rectangle with missing corners) and links (C: component, S:
specialisation, P: precedence/prerequisite, I/P: input/output, R:
regulates/governs, In: instance of). For example, figure 21, shows the
top level of the VisArt model.
The model can be read as follows: The Visart concept (square) is a
component (link C) of both DoCTA and 3 courses. The VisArt concept is
input (link I/P) to two processes (ovals) “designing an instructional
room in TW” and “evaluating the design of an instructional room in
TW”. To indicate that the students are carrying out the design activity, a
principle (student designers) regulates/governs (link R) the “designing
an instructional room in TW” process. Similarly, researchers regulate/
govern the “evaluating the design of an instructional room in TW”
process.
Each of the two processes in figure 21 is expanded at a lower level.
22
Ileana de la Teja, a senior researcher at Télé-université with a lot of experience
in the design and evaluation of telelearning scenarios, served as a consultant and
helped Barbara Wasson model the scenario. Barbara had previous experience in
using MOT from a 1 year visit to Télé-université.
124
Chapter 7 VisArt
Figure 21
Top level of VisArt model
Figure 22 shows the model of “designing an instructional room in
TW”. This model can be read as the “training” process is a prerequisite
(link P) for the “designing an instructional room in TW” process. The
output (link I/P) of the “designing an instructional room in TW” process
is the concept “instructional room in TW”. The “designing an
instructional room in TW” process is regulated/governed by a “deadline” and “evaluation criteria”. There are several inputs (I/P) to the
process including: “tools”, “assistance”, “instructions”, “infrastructure”,
and “student designers”.
125
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 22
A second level of the model of VisArt
Management and Coordination
The completed model of the VisArt activity was used to identify tasks and
to create management and coordination tables from three perspectives:
the instructor perspective, the student perspective, and the researcher
perspective. Each perspective had a table for preparation, training,
design activity, and, if necessary, follow-up. These tables were used to
orchestrate the activity. Some examples are explained below.
Table 21 presents the preparation table from the instructor’s perspective.
Each table has a column for Tasks, a Date the task is to be completed,
Data sources for the task, the State of source, Comments, and who is
Responsible. As can be seen in this table, the instructor has a lot of
preparation tasks. These tasks are shared among the head instructor
(Barbara) and three graduate student assistants (Helge, Trond & Kurt).
126
Chapter 7 VisArt
Table 21 Preparation from the Instructor’s Perspective (as at 24.02.99)
Task
Date
Data source
Prepare
Assignment:
• Course
• VisArt
01.02
Word file,
Prepared
VisArt activity
is accessible
in TW
State of source
Comments
Responsible
VisArt activity Barbara
is part 2; each
instructor to
choose how to use it
Prepare Training 15.02
Activities
Word files,
accessible
in TW
Prepared
Helge
Prepare selfevaluation form
Form on the
web
Prepared
Helge
Prepare Help
15.02
pages on the Web
Web page
Prepared
Kurt
Give out
Assignment
16.02
Assignment
document
Prepared
Set-up TW on
server
08.02
TW
Up & running on
grevling.ifi.uib.no
Set up accounts
in TW
23.02
Participant & TO DO
Team lists
(from Barbara)
15.02
Create classroom 18.02
in TW
Set up Teams
Each instructor Barbara >
to hand out & Arnstein & Lars
go through
with students
Trond
Participant lists Trond
from Lars &
Arnstein
Prepared
Barbara
(w/ Helge
& Kurt)
19.02
Team list (login Prepared
names &
teams)
Received
Barbara
list from
Instructors ;
Assign login
name and team
Create team rooms 22.02
in TW
Team01 to
Prepared
Team11 rooms
in TW
Make sure
Barbara
permissions
are set properly
Ask “drift” for
email aliases
22.02
Team list
Prepared
Email list to
“drift”
Barbara
Inform about
TW license
24.02
License
number
Recieved
Send email to
participants
Barbara
Inform about start 24.02
of the scenario
TO DO
To be sent by
email to all
Barbara
Prepare for
24.02
providing assistance
TO DO
Agree to
assistance
process
Helge w/
Barbara &
Trond
127
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
For example in the first entry it can bee seen that the assignment had to
be prepared (“Prepare Assignment”) by the 1st of February. It had to be a
word file that was accessible in TW and Barbara was responsible to develop
it. When it was complete, the status was changed to Prepared. Eventually,
Barbara would have to “Give out Assignment” and, as indicated, this
would be done by Barbara giving the assignment to Arnstein and Lars the
two instructors at HiNT and HSH, respectively. It was up to each of them
to incorporate the part of the assignment into their course (more on this in
section 7.6). As another example, graduate student Trond was responsible
for the administration of the TW server. There are two tasks related to
this, “Set up the TW server” and “Set up the TW accounts”. Other
important tasks are preparation of the training activities (“Prepare Training Activities”) that were the responsibility of Helge (his Masters thesis
topic) and preparation, by Kurt, of help pages (“Prepare Help pages on
the Web”) on the web. Such an overview of all the tasks made management of the scenario go smoothly. Similar tables, see Appendix J, existed
for the training, design activity and follow-up phases.
As mentioned previously, there also were tables from the student
perspective and the researcher perspective. Table 21 shows the preparation phase from the researcher perspective. The majority of tasks in this
perspective address preparation of evaluation instruments such as preand post-questionnaires, and preparing and getting student consent (to
participate as a research subject). The remainder of the tables that show
the scenario from the researcher and student perspectives can be found
in appendices K and L.
128
Chapter 7 VisArt
Table 22 Preparation from Researcher’s Perspective (as at 24.02.99)
Task
Data source State of source
Comments
Responsible
Prepare computer 01.02
supported
collaborative
learning
assignment
Date
Assignment
file (merge
together
activities)
Part 2 of
assignment
is VisArt
Barbara
Prepare profile
questionnaire
15.02
Profile
Prepared
questionnaire
On web (make Kurt
available 24.02)
Prepare prequestionnaire
15.02
PrePrepared
questionnaire
On web (make Kurt
available 7.03)
Prepare postquestionnaire
15.02
PostPrepared
questionnaire
On web (make Kurt
available 25.03)
Prepare student
consent form
15.02
Consent form Prepared
Send to Lars
and Arnstein
Getting students’
consent
By
24.02
Consent form Prepared
Receive from Barbara
Lars and
Arnstein as well
Assign
researchers to
a team
21.02
Team list
Getting
students’ profile
As
needed
TO DO
Barbara >
Lars & Arnstein
Barbara >
researchers
Prepared
Profile
Prepared
questionnaire
file
Prepare for video 24.02
taping
Prepare for
interviews
Prepared
student >
Kurt
Eskil & Knut
TO DO
Each
researcher
This section has presented the design of VisArt and has shown how we
managed all of the tasks that needed to be coordinated. It is necessary to
spend the needed time on preparing and organising a collaborative
telelearning activity. Otherwise one can run the risk of chaos and
frustration during the deployment of the activity.
Student Consent to Participate in Research
The DoCTA project is registered with NSD (Norwegian Social Science
Data Services) and their regulations for data collection, storage and deletion are being followed. Thus, in order to collect student data in the VisArt
129
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
scenario, the students had to give their consent. A glimpse of the student
consent form for participation in the research project is given below.
The full consent form can be found in Appendix M.
The students were informed about the VisArt activity’s role in the DoCTA
project as follows:
“Our intention is to gain a better understanding of student learning in this
new mode of course delivery, more specifically collaborative learning
over the internet with a distributed team of students. As part of a course
at each of the teaching institutions, a collaborative telelearning activity,
called VisArt, has been designed to have teams comprised of a student
from each of the teaching institutions collaborate through an internetbased groupware system called TeamWave Workplace. Through your
responses to questionnaires, interviews and through electronic data
collection we hope to find more about your opinions of the software, the
activities, collaboration, telelearning in general, etc. There are 8 graduate
student projects associated with VisArt and we hope that you will agree
to provide them with enough data that they can carry out their projects.
(VisArt consent form, see Appendix M)”.
After indicating what and how data would be collected, the consent form
indicates that their participation as a research subject is completely
voluntary and will not have any repercussions on their course grade:
“Your cooperation in the collection of this information is greatly
appreciated, but is completely voluntary. You have the right to choose
not to participate in this research project. Your decision will in no
way affect your grade.
Please indicate by checking in one of the boxes below whether you freely
consent and agree to participate in data collection for research in
collaborative telelearning. If you have any questions regarding these
procedures, please feel free to contact Barbara Wasson at 55 58 41 20
or by email at [email protected]. (VisArt consent form, see Appendix M)”
All students at the three institutes agreed to participate as research
subjects. The collected data will be stored until the end of the project and
then deleted according to NSD guidelines.
130
Chapter 7 VisArt
7.2
The Computer Supported Collaborative
Learning Assignment
As mentioned previously, UiB had responsibility for preparation of the
VisArt scenario. The UiB students would participate in VisArt a graduate
course where the first semester assignment would involve participating
in the VisArt activity. Participation alone, however, was not enough. Since
this is a graduate course and two of the topics the students were learning
about were collaborative learning and telelearning, the VisArt activity
provided an opportunity for both participation in collaborative telelearning
and an opportunity for theoretical reflection. A semester assignment,
found in Appendix N and described in more detail below, was designed
to cover both opportunities. The assignment, which entailed reading CSCL
literature, participating in the VisArt activity, and writing an individual
report that individual report that reflected on the process from both a
practical and theoretical perspective, was adopted at both HiNT and
HSH with minor adjustments (more on this in section 7.6). Each of these
explored in more detail below.
The Background Literature
As computer supported collaborative learning (CSCL) was a topic covered in
all three of the courses, it was natural that the students be given CSCL literature.
The students were told that the following literature should be read:
Dillenbourg, P., Baker, M., Blaye, A. & O’Malley, C. (1995). The
Evolution of Research on Collaborative Learning. In P. Reimann
H. Spada (Eds). Learning in humans and machines. Towards an
interdisciplinary learning science, 189- 211. London: Pergamon.
Koschmann, T. (1997). Paradigm shifts and instructional technology: An introduction. In T. Koschmann (Ed.) CSCL: Theory and
practice
of an emerging paradigm, 1-23. Mahwah, NJ: Lawrence Erlbaum
Associates.
Salomon, G. (1992). What does the design of effective CSCL require
and how do we study its effects? SIGCUE Outlook, Special Issue
on CSCL, 21(3), 62-68.
131
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Gutwin, Carl; Stark, Gwen; Greenberg, Saul: Support for
Workspace Awareness in Educational Groupware. Proceedings of
CSCL’95. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. http:/
/www-cscl95.indiana.edu/cscl95/toc.html.
In addition, a number of web pointers to CSCL related Internet sites
were given, see Appendix N. The students were encouraged to read as
much as possible on CSCL.
The VisArt Activity
The objectives of the VisArt activity were given as follows:
• To provide an opportunity for participation in a distributed collaborative
learning activity where team members do not have the opportunity to
meet face-to-face. This will give experience in both distributed
collaboration and in the use of a groupware system over the Internet.
It will hopefully highlight both positive and negative aspects.
• If each team member has given consent to participating in the research
aspects of this scenario, your team’s experience will also contribute
to furthering our understanding of this new learning phenomenon.
The students were informed that in the VisArt activity they will be part of
a group of 3 students, 1 from Bergen, 1 from HiNT, and 1from HSH,
who will form a design team that will:
• Organise a collaborative team effort
• thinking Salomon’s definition genuine interdependence: 1) sharing
information 2) division of labour 3) joint thinking
• Carry out the Design Activity in TeamWave Workplace (TW) with the
help of:
• help pages (http://www.ifi.uib.no/docta/VisArt/help)
• assistance (email [email protected])
• Help Room in TW
• Produce 2 items:
• A document of your pedagogical decisions (e.g., who is the room
intended for, what is the content, etc….)
• A TW room for teaching/learning
132
Chapter 7 VisArt
The VisArt activity consists of 5 major activities as shown in Table 23. Two
of the activities, pre-questionnaire and post-questionnaire were part of
the researcher’s evaluation of VisArt and are described in section 7.11.
Table 23 The 5 VisArt Activities
VisArt Activity
Date(s)
Preparation
Week 8 (from February 25)
Training on Tools & Collaboration
Pre-questionnaire
23
Design Activity
Post-questionnaire
Week 9 (March 1-5)
March 7
Week 10-12 (March 8-26)
24
by March 26
Preparation. The students were told that in preparation for VisArt they
need to download TeamWave Workplace and some extension files. In
order to make this as easy as possible for the students, Trond Pedersen25 created a web page with directions for downloading. This can be
found at http:/www.ifi.uib.no/docta/teamwave/install.html, see also
Appendix O. In addition, the students were told to use the logon id that
their instructor had given them and were told what their team email was.
They were informed that if they experience any problems they should
contact Helge Underhaug26 by email at [email protected]. If you have
not delivered your consent form to your instructor please do so. If they
had agreed to participate in the research, they were asked to fill in a
participant’s profile questionnaire that can be found at http://
www.ifi.uib.no/docta/VisArt/profile.html and also in Appendix P.
Training. During the training session they were taken through a number
of activities where they gained experience in using the technological tools
and gained experience in collaborating while at the same time getting to
know their team members. A self-evaluation questionnaire (see Appen-
23
24
25
26
Only to be filled in student has consented to participating in the research study.
Only to be filled in student has consented to participating in the research study.
Trond, a graduate student working on DoCTA, had administrative responsibility
for the TW server.
Helge, a graduate student working on DoCTA, had responsibility for designing
the training and served as head assistant during VisArt.
133
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
dix Q and http://www.ifi.uib.no/docta/VisArt/self-evaluation.html) was
provided to help them gauge their competence in using the tools. All
training took place in TeamWave Workplace Training Room. More detail
on the training activities can be found in section 7.5.
Pre-questionnaire. The pre-questionnaire that asks about attitudes
towards TeamWave Workplace so far needed to be filled in before the
design activity began. The questionnaire can be at http://www.ifi.uib.no/
docta/VisArt/pre-questionnaire.html and also in Appendix R.
Design Activity. In the VisArt design activity the teams were to design
a TeamWave Workplace room for learning about some topic. As we
were interested in the process of collaborating – how decisions were
made and carried out, the outcome of this activity included the learning
room as well as documentation about major pedagogical decisions that
were made. More details about the design task were given as follows:
1. The room you chose to design should enable the students to know
more about a concept, a procedure, a theory, a process, etc. Some
examples of possible topics include a room for:
• 6th grade science (e.g., birds, dinosaurs, pollution, etc.)
• high school geometry
• a university course on programming (e.g., recursion, file structures,
etc.)
• a theme in an art history course
• OR a topic approved by the activity leader ([email protected])
2. The room could contain a presentation, a game, a tutorial, a
questionnaire, an exercise, or some combination of these and utilise a
number of the TeamWave Workplace tools. The room you develop
should be under your team’s working room.
3. When documenting the pedagogical decisions that your team makes, you
should include issues such as characteristics of the student audience (e.g.,
age, grade, etc.) and the complexity of the subject area. Create a file PedDecteamx (i.e., use the fileholder tool and substitute your team number in
teamx – team 1 is PedDec-team1) and leave the file in your working room.
134
Chapter 7 VisArt
Post-questionnaire. The post-questionnaire asked about attitudes
towards the training, telelearning and TeamWave Workplace after the
VisArt activity had been completed. It can be found after at http://
www.ifi.uib.no/docta/VisArt/post-questionnaire.html and in Appendix S.
The Individual Report
The individual report is the document the UiB students handed in to be
counted towards their grade for the semester assignments. This is the
part of the assignment that was tailored to fit the courses at HiNT and
HSH (see section 7.6). The students were told that the individual report
should contain:
• an introduction to computer supported collaborative learning and
collaborative telelearning (use literature in class + other information you might find on the web, in the library etc.)
• a description of the design activity you were asked to participate
in, including the tools provided and used (e.g., email, TeamWave
Workplace, etc…)
• a presentation of your team’s room (maybe take a screen snapshot here) and the pedagogical decisions you made to get to the
solution
• a discussion of how the team met Salomon’s requirements for
genuine interdependence and whether or not TeamWave
Workplace supported activities resulting from attempts at meeting
Salomon’s requirements
• a discussion of Gutwin et al.’s awareness concept and what it means
in conjunction with your distributed collaboration through TeamWave
– does TeamWave Workplace support awareness? How?
• your general reaction to collaborative telelearning (as you
experienced it) including: a reflection on the team’s work focusing
on your participation and the others’ participation (did you feel
you did more work, someone did not pull their weight etc. – such
comments will not work against them (you do not need to give
names) and are private to the instructor), the process of carrying
out the assignment, general comments about the entire assignment,
your reaction to TeamWave Workplace (did it help or hinder your
work, was it easy to use, hard to use, etc.)
135
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
7.3
Communication VisArt
Communication was designed to take place directly through TW or through
team email.
In TW, both asynchronous and synchronous communication was possible.
The Chat and Page functions are designed for synchronous communication
(users who are logged onto the TW server at the same time) and tools
such as the MessageBoard, PostIt, Whiteboard are designed for
asynchronous communication.
In order for teams to communicate without logging into the TW server,
each team was given a team email account (on the IFI machines) through
which the team could contact each other. The students were told that
their team email would be collected as data for VisArt evaluation. Thus,
in addition to the 3 team members, the team email included one researcher
who received all the team’s email and was responsible for its archiving.
The head instructor could also use the team email accounts to send
messages about, for example, the start of the scenario, approval of the
choice of domain, etc. In addition, help assistants were available by email.
Assistance in VisArt is discussed in the next section.
7.4
Assistance in VisArt
Another aspect to which a lot of attention was paid was how to provide
help and assistance to the students. We used a model that was introduced
to us by Ileana de la Teja, a senior researcher at Télé-université. Figure
23 presents our design.
136
Chapter 7 VisArt
Figure 23
Organisation of Assistance in VisArt
In this organisation, all help requests go through a head assistant via
email. The head assistant then either answers the help request, or sends
it further to either the content assistant or the technical assistant who
become responsible for answering the question. If the assistant answering
the question feels that the question and answer are relevant to all the
students, the answer can be emailed to all students and not just the student asking the question. Figure 23 illustrates the case where a student
sends an email question to the head assistant who determines that the
mail should be forwarded to the technical assistant. The technical assistant
answers the student’s question directly.
7.5
VisArt in TeamWave Workplace
Included in the design of VisArt was the preparation of the learning environment in TeamWave Workplace. This section presents some of the
rooms that were created for the students including, the Classroom, the
Training Room and the Help Room.
137
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The Classroom
When a student logs onto the TW server, they find themselves in the
Classroom. The Classroom, see figure 24, was created by the head
instructor.
Figure 24
The Classroom in VisArt
In the Classroom there is a To Do List which lists a number to tasks for the
students to carry out (e.g., change their password, add their name to the
Address Book, go to their Team room and leave a message on a PostIt),
etc.). There is an Address Book where students, instructors and researchers
can enter contact information, a class Calendar where important deadlines are indicated (e.g., on March 3 Oppgave 4 legges ut means
Assignment 4 will be available) and a Message Board where messages
can be left for the instructor or researchers. There are also a number of
doorways to other rooms including: the Help Room, the Training Room,
and Team rooms (Team01 – Team11). There is a File Holder that contains
the VisArt assignment (VisArt Oppgave) and four URL links to the profile,
pre-, self-evaluation, and post-questionnaires. Entering the Training room
Doorway, takes one into the Training Room as described below.
138
Chapter 7 VisArt
The Training Room
The Training room, see figure 25, was created and maintained by a
graduate student27 responsible for training and assistance as part of his
Master’s thesis (Underhaug, in preparation).
Figure 25
Training Room in VisArt
In the Training room, there are Doorways to the Classroom and Help Room,
a URL link to the help pages on the web (see section 7.5) and a Self-Evaluation
questionnaire (Egen-evaluering), a PostIt note telling the group to agree on
when they will do the training, Doorways to 4 training rooms (Oppgave 1
room – Oppgave 4 rom) and a Welcome to the Training Room (Velkommen
til Treningsrommet) written directly on the Whiteboard (see figure 26).
27
The training room and majority of the training tasks were designed by Helge
Underhaug.
139
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 26
Welcome to the Training Room
In the Welcome, figure 26, it says:
“The goal of the training period is that you will become comfortable in
the use of TeamWave Workplace and meet the others on your team. In
order for your collaboration to be as best possible later in the scenario, it
is important that you think about the ways in which you can work. This is
a special theme in training assignment 4.”
After the list of available help, the Welcome continues with
“Here you find the doorways to the different assignment rooms. Each
assignment is available in 3 formats MS Word, Mac Word and as text
(.txt, assignment 2 as .rtf). Assignments 1-3 should be finished by March
3rd at 15:00, when Assignment 4 will become available.”
Each of the four assignments was designed for learning either about TW
or about collaboration, and were either an individual assignment (1 & 3)
or a group assignment (2 & 4). Entering Oppgave 1 rom (Assignment 1
room), see figure 27, takes you to the first assignment. Here you are told
to download the version of the assignment you desire and begin. To
download the file you use the right mouse button and click on the version
you want — choose either “Get file” or “View file”. If you finish assignment
1 and want to begin on another individual assignment, you can go to the
Oppgave 3 rom (Assignment 3 room). All of the other three training
rooms are arranged similarly.
140
Chapter 7 VisArt
Figure 27
Training Assignment 1 room in VisArt
Assignment 128 , see figure 28, was designed to help the team members
get to know one another. Each team member was to interview another
team member and present the information for the third member.
Suggestions for which TW tools to use were also given.
28
Norwegian text for assignment 1 and 3-4 can be found in Appendix U.
141
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Assignment 1
Getting to know one another
Each person on the team is to interview one of the other team
members (i.e., A interviews B who interviews C who interviews
After the interview you are to create a textual introduction of your
partner to the 3rd team member (i.e., A writes an introduction of
B, B writes an introduction of C and C writes an introduction of A).
• Carry out the interview as follows:
• Go the your group room.
• Use Calendar to plan the time for the interview.
• Use Chat to carry out the interview.
• Create a ToDoList for your group.
• Present the interview to the others in your group with the File
Viewer.
Create your own individual room (Room, New Room) and name it
the same as your user name. To limit acess to the room, go the the
Permissions menu and set the permissions so that only your team
members can enter your room.
Create a Doorway back to the classroom from your individual room.
Figure 28
Training Assignment 1
Assignment 2 is a team building activity that is based on a personal
development game developed by Dr. Konrad Morgan29 . In the adventure game, Narg Island a shipwreck in a Northern Norwegian fjord is
presented. Teams compete with one another to choose the best ACTION ALTERNATIVES, and SALVAGEABLE ITEMS. An excerpt of the
presentation is given in figure 29.
29
Dr. Morgan, a psychologist, is an associate professor at IFI.
142
Chapter 7 VisArt
Figure 29
An excerpt from the Narg Island Game
Narg Island was adapted for use30 in a distributed environment by the
head instructor, Barbara Wasson. Figure 30 gives the instructions for a
distributed version of Narg Island. These directions were available as a
file in the Oppgave 2 rom (Assignment 2 room). The complete Narg
Island scenario, directions, alternative lists, expert rankings, and scoring
tables are available in Appendix V.
30
With the permission of Dr. Morgan.
143
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Assignment 2
Narg Island Group Activity
The teams will play Narg Island. Here is what you need to do:
a.
b.
c.
d.
e.
f.
g.
First, fetch the scenario (NargIsland-95.doc OR NarIsland-Mac.mcw)
and read it.
Then, fetch a list of ACTION ALTERNATIVES which lists 9 alternatives
and a list of SALVAGEABLE ITEMS which lists15 items. These are Excel
files: ActionAlternatives-95-xls, SalvageableItems-95.xls, OR
ActionAlternatives-Mac. and SalvageableItems- Mac.csv.
Then, Individually and without discussing The Situation, the ACTION
ALTERNATIVES, or the SALVAGEABLE ITEMS, with anyone, rank (1 for
most important, 2 for next most important, etc. with a maximum of 9 for
the ACTION ALTERNATIVES and 15 for the SALVAGEABLE ITEMS)
each according to how important it is to your survival. Fill your rankings in
under STEPs A and I.
Once you have Individually filled in your rankings, arrange a CHAT among all
the team members where you will come up with a team ranking listing your
team decisions under STEPs D and M.
Once you have reached this point, check off on the Meeting roster (in the
TEAM BUILDING ACTIVITY room) that your team has completed the team
rankings. Once all the teams have checked that they are finished, you will be
given the EXPERT RANKINGS which you will fill in under STEPs B, E, J and
N. STEPS C, F, K and O will be calculated automatically for you. You will
probably want to discuss these results with your team.
After this you can check your INDVIDUAL SCORE (STEP Q) and your
TEAM SCORE STEP R). You will need to fetch file Scores1-95.xls or
Scores1-Mac.csv. Hopefully your TEAM SCORE is lower than your INDIVIDUAL SCORE.
If you want to calculate the TEAM EFFECTIVENESS SCORES, arrange to
meet with the team and calculate STEPs S, T, U, V, W and X. You will need to
fetch file Scores2-95.xls or Scores2-Mac.csv.
Figure 30
Directions for a Narg Island Game with Distributed Teams
Assignment 3 is another individual assignment designed to have the
students use the TW tools they most likely had not used yet. Table 24
gives directions for using a number of tools.
144
Chapter 7 VisArt
Table 24 Training Assignment 3
Tool
Task
Web browser
Load a web page into TW.
Try out the web browser.
URL
Use the Help function and find the address of TW’s
own home pages.
Link to them via the URL reference tool.
Post-it
Use a PostIt note to tell your team members what
you think of TW so far.
Vote tool
If you have not already done so, register you vote in
the Classroom.
Meeting roster
Plan an exciting debate or meeting between some
Norwegian politicians.
You decide the theme and goals of the debate/
meeting.
Record these in the meeting roster tool.
Database
Create a simple database to record your own CDs.
Have at least 3 fields (e.g., name, title, nationality).
Legg in 5 album i databasen.
Experiment with the different fields.
Image whiteboard
Draw your house.
Try to import a picture and see you the two can be
combined.
Page
Page a team member and tell that you are finished
with assignment 3.
145
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Assignment 4 was designed to provide an opportunity for a team to
practice collaborating. Figure 31 shows the assignment.
Assignment 4
Collaborating to produce a presentation
In assignment 4 you will collaborate to choose a theme which you
will present for the other groups. You will search the Internet for
web pages on this theme and decide as a group which should be
part of your presentation.
Method for the Assignment:
• Brainstorm about a theme you will make a presentation of.
Use about 30 mintues.
• Individually find web pages that can be used to present your
theme. This part can be done for tomorrow.
• Together as a team, come to a common understanding of
your findings and determine as a group what you should
keep/cast. Use about 1.2 hours.
è Give a short presentation of your theme by linking to the web
pages you have agreed on.
This method meets the requirements for genuine
interdependence as definded by G. Salomon:
1. the need to share information
2. division of labor among the team members
3. pooling together of minds / joint activity of thinking
Salomon argues that for CSCL to succeed, there must be
Figure 31
Training Assignment 4
The Help Room
Another important room we created in TW was the Help Room shown in
figure 32. In this room, the students could find several sources of help. Two
Message Boards — one for content questions and one for technical questions
— were place in the room. Students could either ask or answer questions
146
Chapter 7 VisArt
here. There were also several links to resources on the web. One URL
points to pages on Virtual Teamwork. A pointer on the Whiteboard draws
the student’s attention to the internal TW help system that is accessed though
the Help Menu. This help system is in English. A second URL points to a set
of web page that explain how the different tools and functions in TW work.
These Help pages on the web are explained in the next section.
Figure 32
VisArt’s Help Room in TeamWave
The Help Pages
The versions of TW that we were using until just before VisArt was deployed
did not have any online help. It was decided, therefore, that we would
build our own help pages on the web31 . These web pages would be in
Norwegian and would explain how the different tools and functions in
TW worked. The web pages also gave some tips on how to, for example,
Chat, send a message (Page), etc. Figures 33-36 show some of the pages.
Just before we were to deploy VisArt, a new release of TW became available
and this version included online help. At first it seemed like the efforts to
31
The TeamWave Workplace help pages were developed by Kurt Rysjedal.
147
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
develop our own help pages were wasted, but this was not the case. First,
our web pages were in Norwegian that was preferable for our Norwegianspeaking students. Second, the pages were accessible on the web and a
web window could be open at the same time as a TW window, thus help
was available while they tried to do something. Finally, they could access
the web pages without opening TW, thus giving them a chance to introduce
themselves to TW without having to be on the server.
Figure 33
148
Home Page of TeamWave Workplace Help Pages
(developed for VisArt)
Chapter 7 VisArt
Figure 34
Help page for the Address Book tool
149
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 35
150
Help page for the File Holder tool
Chapter 7 VisArt
Figure 36
Help page for Navigating between rooms
Other screen shots can be seen in Appendix W and the entire help system
can be explored on the web at http://www.ifi.uib.no/docta/VisArt/help.
7.6
Deployment of VisArt
The VisArt activity was deployed for 1 month beginning on February 25th
and ending on March 26th. Following 5 days in which the students were to
download TW, test their accounts and team email addresses, 1 week of
training began. This was followed by three weeks of design activity. This
section addresses this deployment and covers the following topics:
151
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
participants, some reflections on each of the phases of VisArt, assistance
and help, and administration.
Participants
The 32 students participating in the VisArt activity had different
backgrounds, ranged in age from 23 to 68 years, and many had family
responsibilities and full time work. The University of Bergen students
participated in VisArt through the pedagogical information science
graduate course32 “Contemporary Approaches to Pedagogical Information Technology” given in the Spring of 1999. There were 11 students
and their backgrounds were a blend of pedagogical information science
graduate students with a teacher’s background, and information science
graduate students with a social science background. Each student turned
in an individual report as described above.
The five HiNT students who participated in VisArt were students in an
undergraduate course entitled Computer Supported Learning. Four of
them chose to participate so they could use the VisArt experience as a
case in their final exam paper. This final exam paper replaced the individual report that the UiB students had to produce. Two of them wrote a
paper about evaluating TW for pedagogical use. The other two wrote a
paper about differences in computer-supported collaborative work
(CSCW) and face-to-face collaborative work. The last student participated
in VisArt in order to gain experience in using collaborative technologies,
but wrote a final exam paper on a different (unrelated) topic.
The 16 students at HSH Teacher’s College, were senior undergraduate
students training to be teachers who were taking a distance learning course33
on pedagogical information science and were also learning about
collaborative learning. As they were taking a distance course and did not
meet face-to-face, it was possible to have 2 HSH students and 1 Bergen
on 7 of the teams. VisArt was part of the topic Computer Supported
Collaborative Learning (CSCL) and they had to write about their
participation in their electronic workbook34 (elektronisk arbeidsbok).
32
33
34
http://www.ifi.uib.no/staff/barbara/courses/pivar99.html
The course was offered by NVU, the Norwegian Network University (Nettverksuniversitetet) – see http://www.nvu.no/katalog/1999-2000/hsh-alu.html#23
For example see, http://home.sol.no/~aaroenes/Dagbok/CSCL/TeamWave/
TeamWave_4.htm
152
Chapter 7 VisArt
The HiNT instructors35 felt that the design decision to establish teams of
students who were all in geographically different locations was quite important and worked well.
Preparation for VisArt
The preparation for VisArt went smoothly for all but one of the Bergen
students. This student had a MacIntosh and some effort was required to
get his client connected to the UiB server. Due to system constraints at
HiNT, the two instructors had to help with installation of TW. At HiNT the
client software was installed on the private computer account of each
student in order to allow variations (e.g. inserting a portrait/picture) in the
configuration files locally. This was also necessary since the workstations
(PC) were write-protected. Preparation for the HSH students was somewhat
more of a problem as they were only informed of the VisArt scenario when
they got the email (only a day or so in advance) from the head instructor
that the scenario was starting. The students at both Bergen and HiNT had
been told about the upcoming scenario (that there was going to be one,
not what it was) in advance. This lack of notice caused a few problems for
some of the teams in the Training phase.
7.7
Training
HELGE UNDERHAUG
The training-phase lasted for ten days, and the main goal for this activity
was for the students to get to know TW, to get to know the other members
of their team, and to also give them some ideas on how to work and
collaborate in TW. The first problem to occur36 was that not all the students
were given information about the training phase, and thought that the
scenario was to start ten days later. Due to this some students had their
holidays at this time (it was a winter holiday week for Norwegian schools)
and this of course was a problem for the rest of the group. Second, a few
students also had minor difficulties getting attached to the server, and
this also delayed the start of the training for some of the groups.
35
36
Arnstein Edismø and Glenn Munkvold
Revealed through some of the questions on training in the pre-questionnaire.
153
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
An evaluation of the training session is being carried out (see section 7.17)
and will be reported on in a Master’s thesis (Underhaug, in preparation).
From a design perspective, the time and effort that went into preparing
the training for VisArt is immense. Although, two of the four tasks used
in the training phase had been developed and used earlier, they had to
be adapted and prepared for this scenario.
The Design Actvitiy
The VisArt design activity seems to have been successful. The students
were able to collaborate to create a learning room in TW. There were
11 teams and the topics they chose to design learning rooms for included:
endanger species, gothic art, publishing on the internet, triangles, the
big bang, travelling in Denmark, renewable energy sources, between
the world wars, polar bears, and astronomy. Figures 37 and 38 give
examples of the TW working room for Team 7 and their designed learning
room on polar bears 7, respectively.
Figure 37
154
Team 7’s working room
Chapter 7 VisArt
In Team 7’s working room shown in figure 37, it can be seen that the team
was very organised. The VisArt Task (VisArt Oppgave) has a central place
in the room with a doorway to their designed learning room The Polar
Bear – King of the Arctic (Isbjørn – Kongen av Arktis). There are a number
of URL’s pointing to Internet sources of information on polar bears that
were used to design the activities in the learning room (see figure 37).
Figure 38
Team 7’s room for learning about polar bears
In the learning room for polar bear’s (isbjørn) shown in figure 38, the
targeted students are told that they will learn about polar bears. Before
beginning the 4 assignments (oppgave 1 – oppgave 4), they can look at
some pictures on the web (URL Bilder) of polar bears in their natural
habitat. Also they can take a polar bear test (isbjørnn-testen) to check
how much they already know about polar bears. There are 4 assignments
each with 2 or 3 questions that they will find the answers to. For example,
in assignment 2 (oppgave 1) they are to find out “Why don’t polar bears
freeze?” and they are asked to “find their secret to keeping warm”.
Assignment 3 (oppgave 3) is about enemies of the polar bear and the
students are asked to find out how many polar bears there are on the
Norwegian/Russian Island of Svalbard.
155
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Team 10 chose to design a learning room on the topic of Norwegian
authors in the period between the two world wars (Norske forfatterer i
mellomkrigstida). Figures 39-41 illustrate their working process. In figure
39 we see their team room where they have organised the room according
to tasks they have been doing. For example along the left side of the
room we see 4 sets of doorways to (from top to bottom) individual rooms37 ,
training assignment rooms, working rooms where they worked on the
design task, and finally doors to the learning rooms. Figure 40 shows
their working room, which they refer to as their “warehouse” (lager). In
this room we can see that there has been a lot of planning going on.
There are URL links to relevant web pages on Norwegian authors (e.g.,
in the box named I bruk we have Sigrid Undset), a number of sound files
(lydfiler in the circle), File Viewers with illustrations (Illustrasjonar in the
box at the bottom and right of the room), text files (Textar box to the
right of the circle), and so on. It appears that this room has been used to
store possible information sources to be used in their learning room. The
resultant learning room is presented in figure 41.
Figure 39
37
Team 10’s group room
The names of the team members have been “whited-out” to preserve anonymity.
156
Chapter 7 VisArt
Figure 40
Team10’s warehouse
Figure 41
Designed learning room for Team10
157
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Reflections on the Design
There were several weaknesses with the design that were revealed through
formal and informal discussions with the students. One issue was that the
final designed room was not evaluated and that there were so few guidelines as to what should be designed. This decision was made somewhat
consciously in that we were really more interested in the collaboration
process and not so much in the final product. It was assumed that they,
as a team, would decide what to design and this would be documented as
pedagogical decisions. In retrospect, it would have been possible to give
more specific guidelines.
The second issue is that the designed room was just that designed — it
was not deployed with a group of students. Several of the VisArt students
expressed disappointment in that they were not going to get to try their
design out on the students for whom it was designed. This was actually an
unanticipated point. Due to the type of courses VisArt was embedded in
and the time constraints, it was not, nor would it have been possible, to
do so. This is an excellent point, however, when looking at the authenticity
of the design problem. It would have been wonderful to test the designed
rooms on real students, but it would be a challenge and virtually impossible, given the courses we were teaching.
Another weakness of the VisArt design as a whole is that we did not formally
plan for debriefing of the scenario. In Bergen we actually presented VisArt
as part of the DoCTA project after the end of the scenario in order to give
the students insight into the types of research studies we were doing on
VisArt. At this meeting we discussed the reactions to TW, team collaboration
and to the VisArt scenario. At HiNT there was no debriefing although the
instructors feel that there should have been. They feel that it would have
helped the students understand the experience better.
Assistance and Help
From the formal assistant’s perspective, assistance during the Design
Activity proceeded smoothly. The assistants did not feel overburdened
with email questions. This probably was due to the fact that informal,
unplanned assistance was available. During the design phase of VisArt,
we planned for 3 assistants to collaborate to provide assistance/help to
the students. What we did not anticipate was that researchers carrying
out observations in TW, thus “visible” to all on the students logged in at
158
Chapter 7 VisArt
that time, would be asked for assistance in TW – either though the Page
or a Chat function. During the preparation and training phases we tried
to keep track of this activity by using a Message board and Database in
our own researcher’s private room. This quickly became impossible, as
there were just too many requests. In addition, the instructors at the
three institutions were often asked for help when they were logged onto
the system. Details of these situations are being covered in a Master’s
thesis looking at how the instructors and facilitators organise their work
(Wake, in preparation) and summarised in section 7.14.
Even though the majority of the Bergen students were physically situated
in Bergen, little assistance was provided face-to-face. The majority of
assistance was provided in TW. All the HSH students were geographically
dispersed, so all assistance was provided either in TW or through email.
At HiNT, some of the students participating were situated at home while
others were in the computer lab at the college (HiNT) during the whole
simulation. Generally, assistance by the HiNT instructors was given in
the VisArt scenario in the following manner:
• Face to face
• primarily for solving technical issues (e.g. installing TW)
• occasionally when we were in the computer-lab at the same time
as the students did their work
• Email or Chat in TW
• to handle ambiguous elements in the tasks (“What is it we are
actually supposed to do”, “Is our choice of topic OK, we have not
got any feedback from Barbara” , “When is our solution actually
good enough?”)
• to handle administrative issues, for example, some of the students
were planning surveys / questionnaires for the local course
In his usability study, see section 7.11, Rysjedal is concerned with
assistance. In the post-questionnaire, the users were asked to respond to
the following statement: “ If I had a problem, it was easy to find help/
information on how it could be solved.” As figure 42 shows, there were 8
students that agreed, 10 that were neutral to this statement. Three
students disagreed and 3 strongly disagreed. There were none that strongly
agreed. This is not a bad result, but still, it could have been better.
159
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 42
Ease of getting help/information
Rysjedal also found that the message board in the Help Room was nearly
not used at all. His interviews indicate that the on-line help function in
TW was not used very much either. What they used most frequently were
the help pages on the web. These were mainly used in the beginning, to
see how they could install and start using TW. One of the reasons for this
was that they could read the help without being logged into TW. In this
way the help was available in an earlier moment of time then the internal
help in TW. Once they had started to use TW more extensively they felt
that they could get the help they needed by simply asking the other users
logged into TW.
Based on this we can say that it is important that the help is available as
early as possible, and that it should be an external application that the
users can start without having to start the application itself. This is probably
especially important in a groupware system like TW, since some users
might be reluctant to log into an environment with other users present if
they don’t know anything about how the software is working. Actually, it
is possible to start the help in TW without having to start TW itself, but
then you have to find the help-file in the folder where TW is installed.
The problem is that most users will not look in this folder before they get
started. If the help had been available from the TW folder on the startmenu, it would probably have been more frequently used.
160
Chapter 7 VisArt
7.8
Administration of VisArt
RUNE BAGGETUN, ØYVIND MEISTAD, TROND PEDERSEN
This section discusses the administration of, and our experiences with,
the TeamWave Workplace (TW) server.
Downloading, installation. An educational server license for 50 users
was purchased in the fall of 1998, and TeamWave Workplace (TW) version 3.1 were installed on Grevling, one of the Institute of Information
Science’s Unix servers. Grevling is an Axil 320 Sparc Server (NIS+ client).
Students participating in the fall graduate course “Contemporary
Perspectives on Research in Pedagogical Information Technologies”
were given the opportunity to use this groupware application during this
course. This provided us with an opportunity to test TW.
By the time the VisArt scenario was to start in the Spring of 1999,
TeamWave Software Ltd. had released version 4.1 of their product.
TeamWave Workplace version 4.1 was downloaded as an compressed
(gzip, http://www.gnu.org ) archive (tar, http://www.gnu.org ). After
decompression and extraction of the archived file structure, the server
was ready to be started. Administration can be carried out through a
client provided with the server.
Administration. The administrator interface is well arranged and it is
easy to configure the server from this module (see figure 43). Common
tasks include adding/editing user accounts (see figure 44) or groups (see
figure 45), managing rooms on the server and managing the server
repository. The repository is where everything is stored. Whenever
something is added to a room, a reference to it is made here. Versions of
rooms and tools are also stored here. The administrator can selectively
purge rooms and their contents depending upon need. Other screen
shots of the Administration interface can be found in Appendix X.
Inquires from the participants in the VisArt scenario dealt with various
subjects. The most common inquiry was about forgotten passwords, which
was quite easy to solve. Lost data (even complete rooms) was a more
severe problem the administrator had to deal with. A combination of the
version control in every room and the server’s repository module, however,
made a complete recovery of these data (and rooms) possible.
161
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
162
Figure 43
Administrator Interface for server information and sending
messages
Figure 44
Administrator interface for creation of accounts
Chapter 7 VisArt
Figure 45
Administrator interface for creation of groups
Supported platforms. TeamWave Workplace supports various operation
systems, and there is no need to run the client application from the same
platform as the server. In VisArt, most of the students were using Windows
95/98/NT, while the server was running on Solaris. The following platform
are currently supported: Windows 95/98/NT, MacIntosh (Fat binary), Linux
(version 1.2.13; ELF), Solaris (aka SunOS 5.x), SunOS (4.1.4 aka Solaris
1.x), SGI (IRIX 6.2), AIX (version 4.1).
Application size and user installation. Installed and ready to go the
Windows 95/98/NT client version is about 3 megabytes in size. I was
the same for the MacIntosh version that was used by one of the students.
The server application was originally only 1,5 megabytes, but after fifty
users had carried out more than 38000 transactions during the VisArt
scenario, it grew to more than 110 megabytes. Approximately 40
megabytes of this are files (various documents, spreadsheets, etc.) that
the users have uploaded (via the fileholder tool) to the various rooms.
Server stability. Some days into the VisArt scenario we experienced a
reduction in the server’s performance, and one day the server did not
respond to our transaction queries. After confronting TeamWave Soft163
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
ware Ltd. with our problem, we still could not figure out what caused this
to happen. Without going into detail, it was probably an overrun buffer
of some sort. To reduce the poor performance and breakdown of the
server, routines that included restarting the server 3 times a day (at 0900,
1500 and 2000) were initiated38 and carried out for about 1 week.
Restarting the server this way greatly improved the server’s performance.
Eventually ,after unsuccessfully trying to find out from TeamWave why
the server was slowing down, the server was upgraded to version 4.2.
After this there was no longer a need to restart the server unless there
was an explicit need.
In summary, our experiences in administering the TW server are positive
and we intend to continue to use the product in courses at the Institute.
7.9
Evaluation of VisArt
The evaluation of the VisArt scenario is being carried out on several
levels and from several perspectives. In addition to the student’s own
theoretical reflections (described in section 7.10), VisArt is being evaluated
as part of eight Master’s theses. These theses include a usability study of
TW, looking at the efficiency of TW from a qualitative perspective using
the data logs generated by TW, performing a formative evaluation of
how to support collaborative design activities, seeing how TW supports
coordination, how to design training and assistance in a collaborative
telelearning setting, and several activity theory studies of how students,
instructors and facilitators organise their work. In addition, a study on
whether there are any personality factors involved in the use of tools in
telelearning situations has been carried out by one of the researchers.
Numerous data and data collection techniques have been used including
questionnaires, semi-structured interviews, log files produced automatically
by TW, email send among team members, chat files saved by the teams,
documents and artefacts produced by the teams, participant observations,
video taping of activities, self-evaluations and the participants own interpretation of the experience. A number of these studies are summarised
in the following sections.
38
The researchers volunteered to participate in this restarting process. Trond
Pedersen provided directions in how to do so.
164
Chapter 7 VisArt
7.10 The learners’ experiences
A summary of the learners’ experiences as described in their individual
reports is given in the following. In particular, attention is paid to how
their group met Salomon’s criteria for genuine interdependence and
how TW supported Gutwin’s ideas on awareness.
In general, the students were very satisfied with TW. As one student
writes
“An important side with TeamWave is that one can work both
asynchronously and synchronously. … For example one can use
the shared whiteboard synchronously when the users are online at
the same time and write on it together, but it is also possible to use
the whiteboard asynchronously when the different users log on at
different times and work individually on tasks on the whiteboard.
…That it supports both forms of work makes the program package
flexible and accessible at all times.”
Several students wrote that the successful use of TW was not just tied to
the ease of use, rather, that it is used in an activity that meets Salomon’s
requirements. As one student succinctly put it
“I think that a requirement for successful use of it [TW] is that the
participants are motivated and have mindful engagement and that
the tool [TW] is used for something meaningful.”
Sharing of information
The majority of the groups had a heterogeneous makeup with the group
members having different backgrounds. As one group said, this meant
that they had different preconceptions and different experiences with
collaboration. They said, that
“according to Salomon it is exactly these differences that makes
collaboration work…to use each others competence and pull
something useful of these competencies through collaboration.”
When it came to TW supporting the sharing of information, it was possible
to do so both synchronously and asynchronously. One particular group worked
165
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
mostly synchronously (with respect to sharing and they meant this was timeconsuming. Often they would meet at a pre-arranged time and have little to
say to one another and this was a disadvantage as it required that they bind
themselves to meeting at a particular time. It would have been
“better if we could have worked a little bit more asynchronously,
and instead given each other feedback, for example by using a
PostIt, when we were logged on to work with one of our own tasks.
TW provides several tools to support this asynchronous
communication…but our use of this functioned badly”.
Division of labor
With regard to division of labour, one of the students said the following:
“TW was a good support for division of labour in those cases where
it was not necessary to be in our common work area at the same
time. If one person started on a task, then left the work area, the
next person could come in and continue working on the task since
there was a common work area.”
Joint thinking
One of the teams said that the requirement for joint thinking in explicit
terms that can be examined, changed, and elaborated upon by peers was
supported in TW through the Chat, Postit and Brainstorm tools. For them
“This particular need was satisfied through the many online joint
discussions between team members. The fact that team members
had the opportunity to express their ideas, conceptions, meanings,
etc., in clear and observable terms (by means of Postit and Chat
artefacts), enables each one of us to analyze, expand on, or change
these as they were being developed.”
Awareness
The students were in general agreement that TW supported many of the
aspects of awareness as identified by Gutwin et al. (1995). As one student points out, it is TW alone that supported social awareness, but the
combination with some of the training activities that were designed for
166
Chapter 7 VisArt
learning about TW and about each other that supported social awareness.
He says
“The social awareness tied to expected roles is first and foremost
tied to personal relationships. I feel that the training activities we
carried out in TeamWave were important for this. Through informal
chats we were able to each get a feeling of what we could expect
of each other and what type of interaction and tone we would have
between us. … It was never explicitly said that “I am … and you
are …”, but over time we picked up aspects of each others
personaility and found our own roles.”
According to the students TW supported workspace awareness by several
techniques. The Information Areas provided awareness about the other
users presently connected to the server, and indicated who was in the
same room. The coloured cursor on the white board made it easy to
follow what each of the other users who are in the same room are doing,
and if another user is typing on the Whiteboard, PostIt, or Brainstormer,
you see what they are typing immediately. One of the other students
meant that TW’s Chat tool
“provides for some of the social aspects of collaboration. We used
it for informal discussions around different themes, both scientific
and social. Through the common mileu that a room in TeamWave
enables, both task and conceptual awareness are fulfilled.”
Another student said that
“The opportunities provided by TeamWave Workplace in dealing
with the issues in social awareness were generally poor. Support for
interactivity in information exchange and negotiation of roles, which
is fundamental given the interpersonal nature of social awareness,
was limited to artefacts such as Chat and Postit…In the case of concept
and task awareness, the requirement within CSCL environments is
for explicit artefacts that can be used as scaffolds to help students
with task organisation, providde them with concentration aids and
step-by-step guidelines for learning tasks. TeamWave’s support for
both concept and task awareness is provided by tools such as ToDoList
and Concept Map (for task organisation, and Message Board (For
structured message capabilities).”
167
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
It is early in the evaluation of VisArt to make many conclusions. From a
sociocultural research perspective, however, the student’s own reflections
are a very important part of evaluation and as illustrated in this section
they demonstrated an ability to reflect theoretically on practice. The
research reports they submitted in the course contained comments and
reflections that were both thoughtful and insightful and will lead to
improvements in future versions of the scenario.
7.11 A Usability evaluation of TeamWave Workplace
KURT RYSJEDAL
In the usability literature there are various definitions of usability. In this
study, the definition of usability in the Draft for International Standard
ISO/DIS 9241-11 (1996) has been adopted:
“The extent to which a product can be used by specified users
to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use.”
Here the effectiveness of a system relates to the work objectives (or
goals). The efficiency relates to effectiveness in relation to the resources
needed to perform the tasks, and satisfaction, according to ISO 9241,
concerns acceptability and comfort (Lif, 1998). This definition was used
to design instruments to measure the usability of TeamWave Workplace
(TW) in supporting the VisArt collaborative telelearning activity. The
purpose of the evaluation was to identify potential problems with the
system’s usability. Based on the problems found, some implications for
redesign of the system will be proposed. This section reports on this
usability study (Rysjedal, in preparation).
Research design
A usability evaluation of TW should, according to the definition adopted,
include an evaluation of the effectiveness of TW, the efficiency of TW,
and the users’ satisfaction with TW. Such an evaluation has been
undertaken while TW was in use in VisArt, an actual collaborative
telelearning scenario. In VisArt, teams composed of students from three
168
Chapter 7 VisArt
different Norwegian educational institutions collaborated to design an
instructional room in TW.
To guide this evaluation three different questions were asked:
• Can students working in collaboration reach their team goals with
TeamWave Workplace?
• To what extent do the tools in TeamWave Workplace enable students
to meet their tasks?
• Are the students satisfied with TeamWave Workplace?
In order to answer the first question on whether or not students working
in collaboration through TW can reach their team goals, the goals need
to be identified. In this case the team’s main goal was the task objective:
to design an instructional room in TW and to document the team’s
pedagogical decisions. To see if they managed to reach their main goal
entails evaluating the completed instructional room and the document
containing their pedagogical decisions.
To find out to what extent the tools in TW enabled students to meet their
tasks, five of the students were interviewed. The interviews were carried
out after the students had completed their assignment, and an interview
guide (see Appendix Y) was developed to support the interviews. In the
interviews the students were asked about how well they TW supported
the following functions (from Paquette et al., 1995): communication,
administration, production, presentation, and navigation. These are
functions that have to be well supported to make it possible for the students
to meet their tasks in an efficient way. Figure 46 groups the TW tools
according to their functionality.
169
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 46
TeamWave Workplace tools grouped according to
functionality
Questionnaires were used to help determine if the students were satisfied
with TW. In the preparation phase, the students were given a profile
questionnaire (see Appendix P). The profile questionnaire collected
background data on the students (e.g., computer experience, collaboration
experience, educational background, etc.). A pre-questionnaire (see
Appendix R) was used to find out about their first impressions of TW.
This questionnaire was given to the students just after the training phase.
Three weeks later when they had finished their assignment, a postquestionnaire (see Appendix S) was given. The pre- and postquestionnaires together will be used to identify the students’ degree of
satisfaction and see if their opinions changed over time.
All questionnaires were web-based and could be accessed from within
TW by clicking on a URL holder that opened the questionnaire in their
web-browser. This should have made it easy for the students to submit
the questionnaires. They were informed by email when the questionnaire
was accessible and their instructors also told the students that it was
mandatory to answer the questionnaires. Although most of them did answer
170
Chapter 7 VisArt
the questionnaires, some of the students did not submit their answers
when they were supposed to (e.g., one or two students submitted the
pre-questionnaire only days before they submitted the post-questionnaire
— there should have been three weeks between). This made it difficult to
see if their opinions changed over time.
Analysis and Preliminary conclusions
All the data have now been collected, but the analysis has not yet been
finished. Nevertheless, there are some points that can be drawn out of
the analysis so far. This section summarises some of the analyses and
reports some preliminary findings.
Effectiveness
In order to measure the effectiveness of TW in supporting a team’s goals
in the VisArt scenario, the designed TW room and decision file need to
be examined. The completed instructional rooms and documents with
the teams’ pedagogical decisions are preserved on the TW server. In
addition, screenshots of the instructional rooms, both during construction
and after completion have been taken.
With regard to effectiveness, it can most likely be concluded that the
students working in collaboration managed to reach their goal with TW.
All the teams managed to make an instructional room in TW. They also
all made a file documenting their pedagogical decisions. We can, therefore, say that it appears that in this context of use the students managed
to effectively achieve their goals.
Figure 47 gives a screenshot of a completed room and figure 48 shows
an extraction from a pedagogical decision file.
171
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 47
An instructional room designed by Team 6
Figure 47 shows that Team 6 created a project on Publishing on the
Internet (Publisering på Internett). They have created a jigsaw collaborative
task where 76 groups of 3 students solve a problem. A ToDoList (Arbeidsoppgaver) guides them through the activity. Each student in a group
is assigned a number from 1 to 3. All students with the same number are
to go to their room (e.g., Elev 1 rom for student 1, Elev 2 rom for
student 2 and Elev 3 rom for student 3) where they first solve an individual task. Then all number 1 students (likewise all student 2’s and 3’s)
meet to discuss their solutions. Then they return to their own group and
present what they have done. Each group produces a final report on the
web. These reports can be found by following the URL’s Gruppe 1 –
Gruppe 7.
172
Chapter 7 VisArt
PePedagogical decisions:
* OOur target group is students at secondary school.
* We presuppose that the students have already learned to made
web-pages (HTML-programming or use of editiors).
* The teaching material is not based on a single subject, but is an
interdisciplinary task where the students are supposed to gain
some knowledge that later can be used within several subjects.
* References to the curriculum plan: In the curriculum plan for
information-technology in VKI and VKII, presentation of information is an important part of the subject’s definition.
Figure 48
Extracts from Team 6’s pedagogical decisions file
These rooms and documents need to be evaluated as to whether they
meet some criteria39 for an instructional room in TW. As we need to
determine what criteria should have been meet, this evaluation has not
been carried out yet. It will, however, be reported on in (Rysjedal, in
preparation).
Efficiency
The interviews to determine efficiency were carried out just after the
students had finished their assignment. Two of these interviews were
face to face interviews, and three of them were telephone interviews. All
the interviews were recorded and have been transcribed. Analysis,
however, has just begun.
39
There was no specific criteria given in the VisArt assignment, rather the student
team should negotiate their own criteria and document it as pedagogical
decisions. This lack of criteria is one weakness of the assignment design and
should be corrected in future versions.
173
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
One advantage of having the interviews face to face was that it was
easier to see how the students reacted to the different questions—for
example one could see if they had misinterpreted some of the questions.
On the other hand, when doing a telephone interview it was easier to
keep the interview focused according to the interview guide. One problem in all the interviews was that the respondents were very positive.
Observation of the students while they were working on the assignment
had revealed that there were some problems with some of the functionality
of TW. When they were asked about this in the interview, it was expected
that the respondent would mention these problems, but they seldom did
unless they were asked specifically about them.
Pacquette et al.(1995) argue that in order for the students to achieve
their tasks in an efficient way in a telelearning environment, the tools
should support the following functions: communication, administration,
production, presentation, and navigation. In this section only preliminary
results with regard to communication and navigation functionality are
discussed.
Communication. In TW there are tools, see figure 46, for both
synchronous (chat and page) and asynchronous (postit, message board,
vote tool and file holder) communication. The tools for communication
were the tools that were most frequently used during the collaboration,
and also among the tools that the users rated as most important. Still,
the communication tools had some efficiency problems.
In TW tools can be placed in different rooms, thus, the users had to visit
several rooms to see if there were any messages (using the asynchronous
communication tools) for them. This could often take some time, and it
could also lead to communication failure if the teams had not established
some kind of agreement for how and where they would leave their
messages. Figure 49 shows some extracts about what the users said
about the asynchronous communication in the interviews.
174
Chapter 7 VisArt
• It requires that you agree in advance: What will we use to
leave messages. So that we do not leave a message one time
here, and another time there. That we have a standard for
where we should put the message. (…) Maybe we were a little
bit weak on that. We didn’t really have a joint bulletin board
where we left messages. It was a little bit like we… Left
something here and something there. So I guess it could
have been done better.
• I think there should have been a way to leave messages for
people that were offline. That you could reach them, e.g.
with some sort of chat-tool when they weren’t there, like
ICQ. There should be a function that makes it possible to
leave a message for people that are not online and that
comes up as soon as they log on. That would have been nice,
because then you could reach a person. But for the group it
was OK to write on a Postit or in the Calendar, or things like
that. So there were tools for that too, you can say. But I also
missed the possibility to send mail directly.
•
If they should check their mail or something, they had to
check their mail in addition to TeamWave. So it was obvious
that that wasn’t very popular. They wanted all the
communication to be gathered in TeamWave, and not so
much outside. We disagreed there.
Figure 49
Extracts about asynchronous communication in the
interviews
The main tool for synchronous communication in TW is the chat-box
located in each room. In figure 50 we can see some examples of what
the users said in the interviews about the synchronous communication.
As we can see from these examples many users found the chat to be too
slow. The reason for this was that they wanted the tool to transmit
character by character, while this tool only transmitted sentence by
sentence. Actually, many users started using postit (which is originally a
tool for asynchronous communication) instead of chat. Postit transmitted
character by character, and even though this tool had some limitations,
many users preferred it on several occasions. The reason why many
175
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
users preferred the text be transmitted character by character was that
they could see the text while it was written. In this way they were aware
of what the other users were doing. In most cases they could also see
what other users were going to write, so they could start to formulate an
answer before the other user had finished writing.
• The disadvantage with chat is that you have to sit there and
wait and wait, wondering if someone is writing something,
(…) we started to use Postit instead, and then we could talk
simultaneously as much as we wanted.
• When you are using a Postit you could follow what the other
person was writing. You can’t do that in chat. You can see
while he is writing. You get time to think of what is coming.
In this way it promotes understanding, and is quicker.
• We tried for a while to use Postit, but I think it was a bit
annoying that both should write on the same place. Even if
you could see what the other one was writing, you might not
know for sure when the person was finished. You felt that
maybe you broke in… And what is negative with Chat is
that you can’t see. If they write very long sentences in Chat,
it takes a very long time to get an answer on your question,
so then you might just go on. But actually I think it was
better, nevertheless.
• In the Chat-tool, for example, it could have been made so
that several could communicate at the same time. That you
can see what the others are writing, but that it still functions
like a chat. Using some of what you have in the Postit-tool,
in a way. There you can see what the others are writing.
Transfer some of that functionality to Chat.
Figure 50
Extracts about synchronous communication in the usability
interviews
Navigation. Navigation refers to navigation around the rooms in TW.
Most users found that the doorway tool in TW made this navigation very
efficient. By placing doorways between the rooms they used most often,
176
Chapter 7 VisArt
they could navigate between these rooms by just clicking on the doorways.
On the other hand, if they wanted to go to a room that didn’t have a
doorway, they had to locate the room in a list of all the rooms on the
server. This could be more time consuming. Especially as the number of
rooms grew large. As one user phrased it40 :
“It was easy to find them, but after a while the number of
rooms grew very large. So then you used some time to scroll up
and down to find your own rooms. It should have been so that…
I actually don’t know how TeamWave organises the list of rooms,
but it seems very casual.”
Satisfaction
In the questionnaires many of the questions are statements where the
respondents are asked to indicate on a five-point scale to which extent
they agree or disagree, see question 3 in figure 51. There are also many
questions where the respondents are asked to locate their position between
two contrary adjectives on different dimensions. A descriptive analysis has
been carried out on these questions, showing an average distribution on
the different dimensions. On most of these questions the students were
asked to fill in with comments, see question 2 in figure 51. These comments
are grouped together in different categories, which makes it easier to
determine any trends in the comments.
When it comes to satisfaction with TW, the general conclusion that can
be made is that most of the students were very satisfied, but there is still
some room for improvement. In both the pre- and post-questionnaires
the users were asked about their perception of TW regarding usefulness,
simplicity and attractiveness. On each of these dimensions they were
asked to locate themselves on a five-point scale between a set of adjectives
where 1 is the most negative and 5 is the most positive. As we can see
from figure 52, the results were fairly positive. Usefulness and simplicity
attained a mean of 4 or above on both the pre- and post-questionnaire.
Attractiveness was a little bit lower with a mean on 3.61 on the prequestionnaire, and 3.56 on the post-questionnaire.
40
All translations from Norwegian to English have been done by Kurt.
177
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
178
Figure 51
Some example questions from the pre-questionnaire
Figure 52
Perception of TW regarding usefulness, simplicity and
attractiveness.
Chapter 7 VisArt
Some of the other aspects of TW we wanted to know something about were
aspects like TW’s learnability, stability, and if they felt that it was easy to
recover from mistakes. For each aspect the users were asked to which
degree they agreed or disagreed to different statements. Some of the results
we can draw from the answers to these statements are that TW is quite easy
to learn and to use. Still, as we can see from figure 53, there were some
functions that some users found difficult to understand how to use.
Figure 53
Difficulty of understanding functions in TW
Figure 54
Stability of TW server
179
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
There was only one statement regarding stability: ”I think the stability in
TeamWave was good.” This was a statement that most users disagreed
on. We can see from figure 54 that only 5 respondents agreed, while 7
strongly disagreed and 6 disagreed. With this number of users disagreeing
with the statement, we can conclude that the stability in TeamWave
Workplace was not good enough.
There were several statements related to how easy the users felt that
they could recover from mistakes. One of these statements was the
following: “I missed functions like undo and redo.” Even though 46% of
the respondents in a previous statement had agreed that it was easy to
correct mistakes, there was only one that disagreed to this statement
(see figure 55). This indicates that even though many respondents found
it easy to correct mistakes, they still missed functions like undo and redo.
Figure 55
Missing functions in TW
In the post questionnaire the users were also asked to mention 3 negative
and 3 positive aspects about TW. The answers were summarised into one
list of negative aspects and one list of positive aspects. We then ended up
with a list of 54 negative comments of TW, and another list of 56 positive
comments. Many of the comments in the list were similar, or referred to
the same type of problem. The comments in the list were therefore grouped
into different categories according to which type of problem they referred
to. Figure 56 shows which categories the comments were grouped into,
and how many comments there were in each category.
180
Chapter 7 VisArt
Figure 56
Categories of negative and positive comments
Also in this section it was revealed that stability was a problem. In the beginning
of the training phase the server went down a couple of times, but during the
rest of the period the server was restarted regularly to prevent it from going
down while it was in use. Still there are as many as 10 comments that the
stability wasn’t good enough. Related to the comments on stability we have
8 comments on slowness in TW. Several users pointed out that it was slow to
edit, or move things around in the rooms.
There were also several complaints that the tools in TW weren’t good
enough or didn’t work. Many of these comments just state that the tools
weren’t good enough without telling why. Other comments are more
specific and complain that some tools have poor support for other file
formats than .txt and gif. There were also some users who commented
that they missed the opportunity to use sound and video for
communication.
The user interface in TW was an aspect that received both positive and
negative comments from the users. Most of the positive comments were
that it was easy to use and learn. Among the negative comments it is
mentioned that that the display has a tendency to get messy. They also
181
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
found the editing and saving of work to be a bit cumbersome. There
were also some students who felt that the working surface in TW was too
small. This is probably due to the fact that the working surface is dependent on the size of the screen. The smaller screen you have, the smaller
the portion of the room that is visible in the working area.
Conclusions
In this study we have undertaken a usability evaluation of TW. We have
adopted the definition of usability in the Draft for International Standard
ISO/DIS 9241-11 (1996). This definition suggests that a usability
evaluation of TW should include an evaluation of the effectiveness, the
efficiency and the user’s satisfaction.
The analysis of the data in this study has not been finished yet, but the
data that have been analysed so far indicate that in the context of use for
this study TW was an effective tool. All the teams managed to fulfil their
assignment.
To find out how efficient TW was we investigated how well the users felt
they could perform the following functions: communication, administration,
production, presentation, and navigation. Only communication and
navigation have been analysed so far. The users were able to perform
both these functions very well. Still, it was revealed that there was some
room for improvement.
In general the users were also very satisfied with TW. They felt that TW
was easy to use and used little time to learn how to use it. Despite this
general satisfaction there were some negatives. Roughly speaking these
were minor issues, but if they had been dealt with, the users’ satisfaction
would probably be even better.
182
Chapter 7 VisArt
7.12 The Role of Classical Jungian Personality
Factors in Computer Supported Collaborative
Learning Environments
KONRAD MORGAN & MADDY MORGAN
The work described in this section formed part of the evaluation of the
VisArt scenario. Although the VisArt scenario investigated many aspects
about the use of these tools in collaborative telelearning the study reported
in this paper investigated if there were any personality factors involved
in the use of these tools in telelearning situations. It is a summary of a full
report presented in (Morgan & Morgan, 1999).
Review of Previous Personality Research in the use
of predicting and understanding user attitudes and
behaviour
The idea of looking at personality as a factor in human computer interaction
is not new. Van Muylwijk (1983) was among the first to recognise that
personality traits would have a major impact on both behaviour and attitudes
when using technology. This idea of looking at personality and interaction
styles was followed up by both Van der Veer (1985) and Singleton (1989).
Van der Veer proposed a more general approach where various personality
factors would have some affect on attitudes towards and use of the system.
In contrast Singleton proposed intelligence as a single factor which he believed
would predict the degree of success in using computer systems and in having positive attitudes towards computer systems.
Later investigations into the role of personality included a series of
experimental studies by Van-Hoe (1990) who attempted to look at the
role of personality and preferences for menu characteristics and Weil
(1990) who tried to find links to computer phobia. Neither Van-Hoe or
Weil were successful in their attempts to find links between personality
styles and computer use or attitudes. This may have been because they
were simply looking at more high level personality factors or that the
tasks they selected were so far removed from day to day experience that
classical personality theory was overwhelmed by system characteristics.
By the early 1990’s computer systems had advanced enough to allow
for computer mediated communication which permitted more naturalistic
183
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
communication styles. It is not therefore surprising that when Adrianson
(1991) investigated the role of extroversion in computer mediated
communication he found significant differences connected with the
extroversion factor but that these were much weaker than in normal
face to face communication. We must again recognise that in the early
1990’s that although CMC provided naturalistic communication it did
not support video or audio conferencing except in rare research settings.
Previous work by the author
In contrast to some of the less successful studies described in our review,
we have found significant differences in computer attitudes and behaviour
associated with major personality factors (Morgan & MacLeod, 1992).
Specifically we found possible relationships between a stated preference
for command line systems and extreme scores on a high order control
scale, and stated preferences for graphical interfaces and extreme scores
on high order extrovert scales. These promising findings prompted us to
undertake further research, leading to the current study where we
investigated the computer based attitudes and behaviour of students using
computer mediated collaborative learning environments.
The Myers-Briggs Personality Test
The Myers-Briggs Type Indicator (MBTI) is a self-report personality
inventory designed to give people information about their Jungian
psychological type preferences. The measure was developed by Isabel
Briggs Myers and Katherine Cook Briggs in the early 1940s to try and
make C. G. Jung’s theory of human personality understandable and useful
in everyday life, increasing popularity in these types of studies due to its
non judgmental nature.
The MBTI results indicate the respondent’s likely preferences in four
dimensions:
•
•
•
•
Extroversion (E) OR Introversion (I)
Sensing (S) OR Intuition (N)
Thinking (T) OR Feeling (F)
Judging (J) OR Perceiving (P)
184
Chapter 7 VisArt
In general results on the Indicator use letters representing each of the
preferences as indicated above. There are 16 possible ways to combine
the preferences, resulting in 16 MBTI types:
ISTJ, ISTP, ESTP, ESTJ, ISFJ, ISFP, ESFP, ESFJ, INFJ, INFP, ENFP,
ENFJ, INTJ, INTP, ENTP, and ENTJ
Though many factors combine to influence an individual’s behaviours,
values, and attitudes, the four-letter type descriptions summarise
underlying patterns and behaviours common to most people of that type.
The Myers-Briggs personality Type Inventory (MBTI)
in Education
The history of the use of the Myers-Briggs Type Inventory within education
is relatively long. As early as the late 1960’s Richek (1969) had proposed
that the MBTI might be a suitable instrument to determine the best
teachers with regard to teaching style and material presentation to
students. Although the history of the MBTI within education is long it took
some considerable time for it to gain widespread support. Early
evaluations compared the MBTI and other personality measures such as
Cattell’s 16PF in the role of predicting successful learning styles and
grade point averages (Eison, 1985) and although researchers such as
Lorr (1991) had problems recognising the usefulness of the MBTI on the
whole by the early 1990’s there was growing support and recognition
for both the validity and reliability of the MBTI in education (Murray,
1990 ; Brown et al., 1991). Since that time although there have been
some studies which raised concerns that the MBTI was being taken out of
context from Jungian theory (Garden, 1991) and that it might not truly
reflect unconscious desires (Barbuto, 1997) it has been found to be one
of the best predictors for many aspects of education and educational
technology (Eison, 1985; Jackson et al., 1996).
Some of these studies which involved large samples (n > than 1000)
have found the MBTI to be amongst the best personality descriptors and,
reassuringly for the study reported in this paper, that there was no support for the separate gender scoring commonly recommended for the
MBTI (Jackson et al., 1996). This is of importance because the subjects
on the VisArt personality scenario were anonymous with no record of
185
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
their gender, so all subjects were scored in their personality types as
males. It is unlikely given the large meta-analysis performed by Jackson
et al. (1996) that this has any significant implication on our findings.
However, this would need to be confirmed by future replication.
Method
As part of the VisArt scenario within the DoCTA project 26 students were
asked to take part in an evaluation of the TW. This evaluation concentrated
on the use of various component parts within TW. As well as an online
collaborative activity within TW the students were required to complete
three questionnaires, a profile questionnaire, which recorded their
background, their views on group work and their experience with technology. A pre-session questionnaire was also administered which enquired on
the clarity of the task that had been explained to the students, their
preconceptions about TW and the components of the Address Book, the
Brainstormer, Calendar, Concept Map, Database, Doorway, File Holder,
File Viewer, Image Whiteboard, Meeting Roster, Message Board, Post-It,
To Do List, URL Ref, Vote, Web browser, Chat, Page and Whiteboard.
After completing their tasks within TW the subjects were asked to complete
a post-session questionnaire. This questionnaire attempted to record the
student’s views about the clarity of the task they were given and the work
tools. In addition they were asked questions about how effective they felt
the teamwork was within TW and what was the nature of their interactions
with the other subjects within the collaborative working environment.
The findings from these questionnaires are not the topic for this paper
and are dealt with elsewhere, specifically as the Masters dissertation for
Kurt Rysjedal (Rysjedal, in preparation). Instead this study focused on the
correlations which appeared from this data and a personality inventory
(form G Self-Scorable version of the MTBI, 1987) administered to the
subjects prior to the experimental session.
Security and ethical controls
The project described in this section conformed to and was registered
with the Norwegian Data Security Agency (NSD). In accordance with
NSD’s specifications every subject in this study was assigned a unique
186
Chapter 7 VisArt
subject number. These same subject numbers were used to identify their
behaviour online in the TW environment and their responses to the profile, pre-session and post-session questionnaires. At no time did the
experimenter know the identities of the subjects.
Data entry and analysis
The MBTI personality surveys were hand coded according to the self assessment schemas provided by the Myers Briggs corporation. These codings
resulted in a raw score in each of the major dimensions of personality
recorded by the Myers Briggs personality test, namely, extroversion (E) or
introversion (I) Sensing (S) or Intuition (N), Thinking (T) or Feeling (F), Judging
(J) or Perceiving (P). Normally these raw scores are converted into a single
set of four primary personality measures such that the respondent is provided
with a four-letter combination representing their personality preferences.
As we described in the review of the Myers Briggs personality test above,
this process leads to 16 possible ways to combine the preferences and
result therefore in the 16 MBTI types. Rather than using these simple
letter codings we took the raw numerical scores in the primary eight
preferences and used these to provide each respondent within our survey
with a score for their preferences in extroversion, introversion, sensing,
intuition, thinking, feeling, judging or perceiving.
We did this because the simple letter types lose the strength or degree of
the respondent’s preference on the four major personality dimensions
recorded by the Myers Briggs personality test. It is also possible within
the Myers Briggs personality test for an individual to be rated for example
as an introvert (I) when in fact their extroversion score was nearly balanced
to their introversion score. This is because a single number difference
results in the larger score being taken as the primary personality dimension. By just using the raw scores within the four dimensions we therefore
retain the original information about the respondents personality. After
the data had been manually decoded into the primary scores of the four
dimensions of the Myers Briggs personality inventory, they were then
entered into a statistical package (Minitab v.10) combined with the data
from the profile, pre-session and post-session questionnaires. An
exploratory analysis was then performed on the data looking for significant
correlations, using a Pearson correlation test, between the primary eight
measures of personality provided by the MBTI.
187
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Results
In the following results section we will discuss the findings from the Pearson
correlations between the eight primary MBTI types and the responses
from the profile, pre-session and post-session questionnaire data. Within
each of the questionnaires we will summarise the findings associated with
four primary personality dimensions.
Summary of Findings
Within our study we appear to have found support for the theoretical
involvement of personality factors within behaviour and attitudes in computer mediated collaborative working environments. Our work is based
upon a relatively small sample (n = 26) and therefore our findings can
only be preliminary and exploratory within the area of personality and
computerised behaviour and attitudes. However, we do appear to have
found some strong and cohesive relationships between Jungian personality
factors and human behaviour in a novel environment. When we examine
our findings we can see clear patterns within each of the major personality
dimensions. It does not appear unlikely that these differences would follow
over into predictable generic guidelines for the creation and use of
collaborative computer based learning systems.
In particular the major difference between Extrovert and Introvert seems
to have been extremely clear within our sample such that Extroverts
were actively looking forward to the group activity before their session
while Introverts reported strong negative anticipation.
The importance of social interaction in such a collaborative learning environment cannot be overestimated and while it was predictable that
Extroverts would be intensive Chat users and Introverts be reluctant or
infrequent Chat users it was an interesting finding that Extroverts were
more likely to have their own website and to have significantly more
positive views towards the use of web browsers. The social interaction
differences between Extroverts and Introverts was also found to be
reflected in their preferred learning style such that Extroverts found the
social part of the interaction between student and teacher to be a vital
part of their learning experience whereas Introverts clearly preferred to
work and learn alone. This difference was reflected not only in humanhuman interaction (teacher- student) but also was hinted at in terms of
188
Chapter 7 VisArt
artificial agents within the Help support of TW so that Extroverts found
such agent based Help systems to be very useful and Introverts found
them to be unhelpful. These fundamental differences meant that TW
was predominantly geared or designed towards an Extrovert based
learning style and may have disadvantaged the Introvert students so that
the Introverts reported having problems learning how to use TW and felt
that TW was unstable. In contrast Extroverts felt that TW was easy to use
and quick to learn. As a possible side effect to the match between TW
and the Extroverts learning style they reported finding TW significantly
more stable as a product than their comparative Introvert users.
Within the Sensor/ Intuitive dimension (S/N MBTI types) we found a
significant difference in the reported attitude and use of several of the
TW tools. In addition to finding TW difficult to learn the Sensors also
avoided using the Message Board, Post It Notes and Chat tools preferring
instead to use the Address Book, BrainStorming, Calendar, Concept Map
and Database. In total contrast those individuals who were high on the
Intuitive dimension often used the Chat system and thought that TW was
enjoyable and easy to learn. Although these Intuitive individuals used the
Post It tool frequently they tended to significantly avoid the use of the
favoured tools of the Sensor.
Given these findings future development of computer based collaborative
learning environments might find profitable research in investigating the
effect of adapting the tools based on the preferences of individuals at the
extremes of the S/N dimension. From the evidence we have seen in this
study there appears to be a clear split between tools which support
individuals high on the S scale and those high on the N scale. Only future
studies of these factors will be able to isolate the specific aspect of each
of these tools responsible for the differences we have found.
The remaining two dimensions of Thinker/Feeler and Judger/Perceiver
appear more concerned with ways in which group work is viewed and
performed. However we should be cautious of the fact that it appears
that individuals high on the Thinker end of the Thinker/Feeler scale and
the Perceiver end of the Judger/Perceiver scale were older and
significantly more experienced in group work. It might be that the
differences in attitudes towards group work reported within our study
were a reflection of the age and prior experience of our subject
population. Given that possibility it does appear that Thinkers feel that
group work involves more effort and greater responsibility than their
189
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Feeler co-workers. Likewise individuals high on the Judger scale tended
to enjoy group work within TW while Perceivers, who were more
experienced of group work, disliked the idea and experience of group
working in TW and felt that luck was more of an element in the success
of such collaborative efforts.
Conclusions
In this study we have investigated the possible link between MBTI types
and the attitudes and behaviour of students using computer supported
collaborative learning environments. We have found statistically significant
differences between the major personality factors in terms of the use of
the learning environment and attitudes towards the various components
of the online collaborative learning system. Based on our findings we
strongly support the idea that the MBTI can provide a useful tool in
configuring such online learning environments to student’s personalities
and preferred learning styles.
Of the MBTI types investigated in our study the dimensions of Extrovert/
Introvert and Sensor/Intuitive appear to be most promising as major
predicting factors in learning styles and system component preferences.
These can be summarised such that Extrovert / Introvert dimension
appears very useful in determining primary learning and interacting style
and preference and the Sensing / Intuitive dimension appears most useful
as a predictive tool for the use or avoidance of certain system components.
The remaining dimensions Thinkers / Feelers and Judgers / Perceivers
appear more related to attitudes toward group work and may reflect
some previous experience within our subject population.
Although Myers-Briggs and Jung did not include maturity of the individual within their personality types in our subject population there does
appear to be a strong trend within the Thinker/Feeler and Judger/
Perceiver dimensions correlated with greater age and experience. Within
our sample the older and more experienced individuals disliked group
work and were more likely to report system based problems linked to
group based activities. It is uncertain if these findings related to the
Thinker/Feeler and Judger /Perceiver scales will be reflected in a more
general subject population.
As a result of this study it is to be hoped that future research will investigate
methods in which MBTI type tests for Extrovert / Introvert and Sensor/
190
Chapter 7 VisArt
Intuitive can be directly linked to real time online changes within the
collaborative learning environment to more closely match students
preferred interaction styles and preferred learning tools.
7.13 Evaluating How Students Organise their Work
ESKIL ANDREASSEN
This Activity Theory-oriented evaluation has been carried out as a part
of the VisArt scenario and is being written as a Master’s dissertation
(Andreassen, in prepraration). More specifically, the subject of this study
is the activity undertaken by the students in Team3, designing a TW
room for teaching 6th graders about endangered animal species, a task41
of their own choice, within the boundaries of the VisArt design activity.
The data collection was carried out between March 1st and April 11th.
The students were geographically distributed over Norway and were not
able to meet face-to-face. This distributed nature of the collaboration
process influenced the way the students collaborated, and an important
part of the study is to document how they organised their work in order
to overcome this challenge.
The research question is:
How do the students organise their work?
This tentative research question, whose scope has been narrowed down
during the data gathering, will be specified further during the data analysis.
The focus will be on trying to detect how the participants co-ordinated
their activities, how they divided the labour between them, and what
kind of tools that were used in the collaboration process.
41
“Task” is to be understood as the design task the students chose themselves, the
task of designing a TW-room for teaching 6th grades about endangered animal
species. The “activity” is the process of collaboration running from March 1 st to
April 11th, i.e. the period framing the first and last contact between the students.
191
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Evaluation approach
The goal of the evaluation has been to conduct a naturalistic study of
how participants in collaborative telelearning organise their work and
learning activities (Guribye & Wasson, 1999). An ethnographic approach
was chosen for three different reasons. There is already an emerging
tradition of using qualitative methods or a naturalistic approach (Savenye & Robinson, 1997) to investigate different phenomenon within the
area of Computer Supported Collaborative Learning (CSCL) (Koschmann,
1997). Also, the choice of Activity Theory (Leont’ev, 1978, Engeström,
1987) as theoretical approach in this study calls for the use of qualitative
methods of data collection (Jonassen & Rohrer-Murphy, 1999, see also
Nardi, 1996). In addition, Yin (1994) suggests, that “when asking a “how”
or “why” question about contemporary events with little or no control
over the behavioural events (p. 9)”, an exploratory case study design is
called for. These taken together, and given that the goal is to understand
how the students organise their work, and to see what tacit practices that
underlie the accomplishment of work practises, the use of an ethnographic
study could be fruitful (Pettinari & Heath, 1998).
Ethnography (see Hammersley & Atkinson, 1983, Denzin & Lincoln,
1994; Harper, 1998) influences the design of the evaluation approach
including the choice of data collection methods and analysis techniques,
in other words, the evaluation is an iterative process where an ongoing
analysis guides the data collection emphasis in successive phases.
The data collection
During the scenario, different data gathering techniques were used, providing different data sources. The most important technique was electronic
collection of communication, both emails and logs of TW chat-sessions.
The students had a total of nine TW chat sessions, five in which all three
students took part, four in which only two participated. Seven of the nine
chat sessions were held before the students decided on the task, only two
after. This, which can be seen as a shift in communication mode, is
discussed below.
The TW chat logs were saved and forwarded by the students. There
seem to be a few “holes” in the chat logs, i.e. parts of logs are missing.
This is mainly due to the fact that the students forgot to save the chats
192
Chapter 7 VisArt
when moving from one TW room to another, or that they experienced
technical problems when saving. These “holes”, of course, might influence
the validity of the data in that it can weaken the understanding of the
collaboration process as such.
To observe the students collaborating, immersing oneself in the virtual
environment, in this case entering the TW room during sessions without
directly interfering or taking part in the work, provided a good opportunity
for taking field notes and watching the students “in action”, deepening
the understanding of the process. A non-participatory role, a sort of
passive presence, was adopted when observing the students, and the
actual interaction with the students was minimal, limited to one question
from one student regarding the saving of chat logs. Whether this passive
presence influenced the students in a substantial way is hard to say, but
when asked in the post scenario interviews, they stated this presence
was of little or no influence on their work and collaboration effort.
A total of 20 emails were sent between the students, nine using the team
address (automatically forwarded), and eleven between two students
without a copy to the third person. After the scenario had finished, mail
that was originally sent without using the team address was, by request,
forwarded by the students. Based on the reconstruction of the mail
communication, it is reason to believe that all emails have been obtained.
The kind of mail being sent, addressed to the team or to an individual,
was evenly dispersed throughout the scenario.
Post scenario interviews with all three students, one face-to-face and two
telephone interviews, were also conducted. The interview guide (see
Appendix Z) that was based on the preliminary findings of the data analysis
and phenomenon observed during the data collection, loosely guided
these interviews. The choice of Activity Theory as a theoretical framework,
emphasising aspects like, division of labour, tool mediation, and rules,
also made suggestions about what to look for during the data gathering
and what to ask when interviewing the students. Informal conversations
with the student situated in Bergen during the scenario provided additional
information about the collaboration process, allowing for clarifying
questions, and inquiries about the work already carried out, as well as
the planned progress, to be made.
The interviews were a valuable supplement to the data gathered during
the scenario in that they presented the opportunity to ask questions to
193
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
clarify indistinct aspects of the collaboration effort. In addition, the student brought up new themes that helped in understanding the process.
The data
Qualitative methods, or a naturalistic approach, have been dominant in
investigating the collaboration process. The data collected consist of chat
logs from TW sessions, emails, transcribed interviews, and artefacts
created in TW by the students.
In addition, the transcripts of the electronically collected log files from
TW Workplace, which includes periodic, chronological recordings of all
artefacts in the environment, as well as statistical data of who was logged
on when, have been made available. This might turn out to be valuable
supplementary information in trying to understand the “time aspect” of
the collaboration process; who was logged on when, were they logged on
at the same time, and, if they were, what rooms did they work in, what
changes did they make?
The techniques, how the data were gathered, and the kinds of data
produced are summarised in table 25, below.
Table 25 Data gathering techniques
Data gathering
Techniques
Gathered by
Type of data
Electronic collection of chat logs Saved and sent by students Chat logs
Field observation during
TeamWave sessions
Taking field notes during
observation
Field notes
Electronic collection of email
Received automatically as
a member of the team
mailing list
Emails
Informal interviews
Recorded and later
transcribed
Transcripts of
interviews
Collection of artefacts
produced by the students
Saved or received by
researcher
Artefacts and
documents produced
TeamWave logs
Collected electronically
Transcripts of
TeamWave logs
194
Chapter 7 VisArt
Preliminary data analysis
As stated above, the focus of this study is on how the students organised
their work. From an activity theoretical perspective the division of labour, tool mediation, societal rules, and the object of the activity are all
important components of an activity.
One can organise these components of an activity, like the collaboration
process in VisArt, into an activity system (Engeström, 1987). This activity
system can be used in the process of developing a deeper understanding
of the activities under study (Bakardjieva, 1998). In figure 57, below, the
team’s effort of producing a TW room for teaching 6th graders about
endangered animal species, has been put into such an activity system.
Figure 57
Producing a TW learning room as an activity system
When looking for ways an activity such as this has been organised, the activity
system might give clues about what aspects to emphasise. Questions such
as: How did the students co-ordinate their work? In what way did they divide
the labour? What rules facilitated, or constrained their activities? How did
they take the community into account? How did they decide on the design
task?, Did the goals and motives change in the course of the scenario? A few
of these aspects will be treated more thoroughly in the following sections.
195
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The collaboration process can be divided into four phases:
• Phase 1: the VisArt training period (March 1st to 7th)
• Phase 2: the first part of the Design Activity, before the students
decided on their task ( March 8th to 12th )
• Phase 3: the latter part of the Design Activity, i.e. from having decided
on the task till the end of the scenario (March 13th to 26th)
• Phase 4: the time after the scenario had ended (March 27th to April 11th).
Co-ordination
In order to finish the assignment the students had to work together, and
hence, had to co-ordinate their work. Co-ordination is thus to be
understood as the efforts undertaken by the students in order to act
together as smoothly as possible. This includes appointing meetings in
TW, updating each other on new or changed appointments, dividing the
labour, and keeping each other posted on who has done what.
During the second phase of the scenario, before they decided on the
design task, teaching 6th graders on endangered animal species, the
students met regularly for synchronous collaboration sessions. The
meetings were arranged by using mainly email, but TW was also used to
make sure that everybody was notified42 .
Email from Lucy
I have made a note on the Message Board in Team03 suggesting that we meet to agree on how to solve the assignment. My
proposal is 6 PM tomorrow Tuesday March 9th. We have three
weeks to finish the task, or am I wrong?
This kind of “double communication” occurred in phase one and the
initial part of phase two, and was not used in the latter parts of the
process. Given the fact that this double communication only occurred in
the initial stages of the activity, one possible explanation is a lack of
mutual understanding of, or mutual agreement on, forms of collaboration.
Not knowing whether everybody would log on to TW before the suggested meeting, sending a mail in addition to the note on the message
board, could act as a reassurance that everybody was notified.
196
Chapter 7 VisArt
Later, new meetings in TW were arranged at the end of each session,
Lucy: I suggest that we work individually and that we meet
Monday or Thursday morning.
Anna: Tomorrow?
Lucy: Yes, if we must meet before Monday at 5 PM?
Michael: I think tomorrow is too soon. I’ll go for Monday at 5 PM.
Anna: (………) Monday is better for me too.
Lucy: OK, Monday it is.
If , for some reason, one student was not present in the chat sessions or
the meetings had to be rearranged, the students left messages in their
TW-room, or sent emails, as a standalone action, not as double
communication.
Another possible interpretation of the disappearance of the double
communication is that the students developed an increased insight in the
technical functioning of TW and email as the scenario ran. One example
of lack of technical understanding is a reply made by Lucy to an email
sent by Michael, using the team email address (which had all the
participants as recipients).
Hello. I hope you are feeling better. Anna and I have done
training task 1 and 2, the 2nd and 3rd of March. I will place a
note for Anna in our TeamWave room telling her that you have
sent me an email. I’ll see you in the Team03 room in
TeamWave.
Lucy clearly does not understand the function of the team3 address,
making an additional note in the Team03 room, informing Anna of
Michael’s mail. Although this does not go directly to “double
communication” it may be an indicator of poor technical understanding,
leading to double communication.
The establishment of routines for collaborating, combined with increased
technical understanding and skills, may be possible explanations for the
absence of double communication in the latter parts of the activity.
42
Eskil has translated this extract, and all the other extracts. For the sake of
anonymity, the names of the students are fictitious.
197
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Communication mode
Also, the collaboration mode seemed to shift from synchronous to
asynchronous as the scenario went on, with a noticeable change before
and after the decision on the task. As Anna expressed it in a post-it note
in TW, two days after the decision was made:
It’s really not necessary that we meet again until we have done
what we agreed on Monday, is it? We don’t have to be online at
the same time to work on our rooms. We can look at what the
others have done when we are logged on, and leave a post-it, like
this one, if we want to comment on anything. What do you think?
And, when being asked during an interview, what she thought of the
synchronous meetings on TW, she elaborated on the same subject.
I don’t think it worked out at all, we met to often, (………), is it
really that important to be online on the same time to discuss
what to do? Of course, when deciding on the task, we had to be
online using the brainstorm-tool, but in the project itself, we
could have managed with a Message Board, we could leave
messages and make comments. At least that’s what I think.
After deciding on the task, the students divided the labour between them,
each being responsible for one part. Their idea was that they should
work on their own creating the rooms, and that the others should drop in
and comment on that work that had been done, and propose changes
they thought should be made. This may be a possible explanation to the
shift in communication mode, also suggested in the post-it note and extract
from the interview with Anna, above.
Division of labour
After deciding on “splitting” the task, and such, the world into six parts,
the five continents and Norway, they divided the task between them in
the following way:
Michael: I don’t want to do Norway!
Lucy: I can do Norway (I’ll see what I can find)
Michael: I can do Africa
198
Chapter 7 VisArt
Anna: I’d like to do Asia and America
Michael: Norway may represent Europe?
Lucy: We agreed on being more thorough on Norway. I guess
that part is enough for one.
Michael: Yes.
Michael: I can do Africa + Australia
This part of the chat session reflects the actual division of labour, but the main
reasons for this division, and which student was assigned which part, are to be
found in the part of the chat where the students are trying to decide and agree
on the task. Lucy expresses an interest in Norway, and what she calls “social
conditions”, and elaborates on her interest in the interview;
(……) I wanted to take more than the science part into
consideration, you know animals and things like that, that it
has a social side. I live in Nord-Trøndelag and it has been quite
a few discussions via the media concerning “predator policy”
and things like that;
and goes on:
(……) I found that it was okay to relate it to Norwegian
conditions, because I had nothing to contribute with in relation
to other parts of the world.
This does not go directly to the division of labour, but may be regarded as
one aspect playing a part in the assignments of work areas, in that it
reflects what can be termed as “personal interests”. Lucy being the only
one expressing a wish about what she wanted to do, was granted her
wish. The other students also gave answers to questions regarding the
division of labour, that might support such an interpretation.
EA: How did you divide the labour between you?
Michael: It went by itself really, you know…, it depended on
interest, maybe. It went really fast, dividing the labour, it
presented no problem
EA: So it was based on personal interest and…
Michael: Yes
EA: So there were no discussions as to who should do what?
Michael: No
199
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Although not very extensive, Michael’s answers might point in the same
direction as Lucy’s statements, that the assignment of subtasks depended
on personal interests. Anna also made similar claims, but she, in addition,
commented on what she saw as the reason for the physical division of the
task into subtasks, each student being assigned two parts of the world:
(……) and we found out that if we were to divide the task into
six parts, six parts of the world, we could do two each. And
then be given free leash within those two rooms, and then we
could do the main room jointly, or how we wanted, in this
manner of speaking it was a pretty okay division of labour. At
least I think so.
This division of labour may have contributed to a co-operative form of
work, rather than a collaborative form, (cf. Dillenbourg, Baker, Blaye &
O’Malley, 1996), in that the work was “divided among the participants
each being responsible for a portion of the problem solving, rather than
the work being a mutual engagement of all participants in a co-ordinated
effort to solve the problem together” (Roschelle & Teasley, in press,
cited in Dillenbourg, Baker, Blaye & O’Malley, 1996:190).
This division of labour, inviting to a co-operative kind of work, may also
have played a part as the communication mode shifted from synchronous
to asynchronous in the course of the scenario. The students did not have
to work synchronously, as they were to produce a room on their own, not
as a joint effort.
Although the division of labour opened for individual problem solving,
the students recognised the value of feedback and agreed on providing
this during the scenario.
Feedback
In spite of their agreement on helping each other and commenting on
each other’s work, feedback hardly ever occurred. When being asked
why this didn’t happen, the students answered that it was mainly because
they wanted to finish the assignment as fast as possible, and that when
they didn’t get any response when actually trying to comment on the
work, they soon gave up. Lucy answered, when asked if the others
commented on her work:
200
Chapter 7 VisArt
No, they didn’t. I only felt that when we were finished, that
they found it OK. But I really didn’t expect a lot of comments
during the process anyway, because I believed that they felt
that they had enough with their own part. But I looked at what
they had done, because I felt I had to, ha-ha, to say what I
thought about it (……)
Anna said that she tried to make comments, but that she never “got any
response” and that “ no one ever commented on her work”. Michael
claimed, as stated above, that he liked working on his own when designing “his room”, but that he could have wanted to meet the others more
frequently during the third phase of the scenario “to get some feedback” on his work.
A possible explanation for this lack of feedback can be found in the
interview with Michael:
EA: Why didn’t you do that? (Give feedback)
Michael: It was time pressure I guess. That we were to be finished
by a certain day
Anna’s statement in a interview may support such an interpretation;
Anna: (… …) maybe the others also felt that it was to time
consuming, that one had to concentrate on one self rather than
the others, yes.
They, again, ascribe this time pressure both to the new TW mediated
way of collaborating, which they found time consuming, and the size of
the task, which they found to large, and therefore reduced in the last
days before the deadline of the design activity.
Summary and preliminary findings
In order to study and elucidate the organising of the VisArt collaborative
activity of Team3, a naturalistic evaluation approach has been applied.
Different qualitative data gathering techniques have been used providing a variety of data sources, ranging from electronically collected TW
chat logs to transcribed informal interviews.
201
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The preliminary data analysis has dealt with aspects like co-ordination,
communication mode, division of labour, and feedback. During the entire
scenario the students met regularly in TW, and co-ordinated their actions
by using TW or email. Sometimes both TW and email were used as a
means of co-ordination, providing a form of double communication. This
form of communication disappeared as the scenario went on, maybe as
a result of the establishment of regular meetings and patterns for
collaboration.
The task decision marked a noticeable line of demarcation in the
communication mode. Before deciding on the task the rate of synchronous
meetings and communication were higher than after the decision had
been made. The asynchronous nature of the post decision work, may
have its root both in the fact that the need for synchronous meetings
were diminished, and that each student was assigned her/his own area
of responsibility, contributing to a co-operative, rather than a collaborative
form of work. In spite of a mutual agreement on providing feedback on
each other’s work, this hardly ever occurred. Time pressure and a feeling
that one had to concentrate on what oneself was doing, are probably the
main reasons for the lack of feedback.
7.14 Evaluating the Organisation of a Collaborative
Telelearning Scenario
JOWAKE
The particular perspective in this Master’s study (Wake, in preparation)
is to evaluate the VisArt scenario from the instructor and facilitator
viewpoint. More specifically, an understanding of how the instructors
and facilitators organise their work is pursued. Activity Theory (Leontev,
1978, Engestrøm, 1987) is used as a conceptual framework to guide
the data collection and analysis.
Methodological aspects
The data gathering techniques employed to address the research question
can be characterised as qualitative methods. The reason for taking this
particular approach, is partly justified by the theoretical foundations in
202
Chapter 7 VisArt
the DoCTA project, partly by the phenomenon at hand, and mostly by
the research question. First, the theoretical perspective chosen, Activity
Theory, necessitates qualitative methods of data collection (Jonassen,
1998). There is also an emerging tradition of using qualitative methods,
or a naturalistic approach, to investigate phenomenon within the field of
CSCL (Savenye & Robinson, 1997). Second, the research focus on the
intra-group collaboration of a group of six instructors/facilitators — or
the people and artefacts in interaction — suggests that an ethnographic
approach might be useful (Pettinary & Heath, 1998).
The data gather techniques used were semi-structured interviews and
collection of mail sent between the instructors and fascilitators during the
scenario. Participant observation was also used to a certain extent, and
the use of TW log files is currently being considered. Interviews of all the
five instructors/facilitators that were assigned a formal and pre-planned
role in the scenario were undertaken, and their responsibilities can best
be characterised as “administratively responsible”, “technically
responsible”, or “responsible for training”. In addition to the head instructor
in Bergen, there were three, instructors located at HSH and HiNT43
who were responsible for integrating the collaborative telelearning activity
into their course, and preparing their students for participation in VisArt.
These roles are summarised in Table 26.
Table 26 Instructor and facilitator roles in VisArt
Formal role
Main, pre-planned responsibility
Head instructor
Responsible for the overall administration of the scenario.
Help/ training facilitator
Responsible for training the students before the scenario,
and receiving/answering/forwarding requests for
assistance from the students during the scenario.
Technical facilitator
Responsible for maintaining the TW server, and helping
the students with technical problems.
HiNT instructor
Responsible for preparing the students at HiNT for the
scenario, and integrating it into the semester courses for
his students.
Stord instructor
Responsible for preparing the students at HiNT for the
scenario, and integrating it into the semester courses for
his students.
43
Although there was only one formal instructor role at HiNT, the role was actually
shared by two persons.
203
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Semi-structured interviews were performed using an interview guide (see
appendix AA), that was designed to reveal the particular instructor’s
views or opinions. The theoretical perspective chosen largely influenced
the questions asked. The same questions were, in general, asked to all
the instructors/facilitators. The questions investigated the division of labour, the tools or vehicles of communication and work organisation, the
conception of roles and goals, and any sociocultural rules or norms. Information about the particular social context in which the learning activity
took place was also gathered.
To gain an understanding of how activities in VisArt were organised, all
the email, sent between the students, instructors, and facilitators during
the scenario was collected. As described in an earlier section, the design
of assistance designated one head assistant, referred to here as the help/
training facilitator. This person was responsible for preparing the students
for the scenario by training them in use of TW and in coordination and in
getting them acquainted with each other. The help/training facilitator
was also designated to receive all mail queries from the students who
had questions. If the help/training facilitator could not answer a question,
he categorised it passed on to either the technical expert (technical
facilitator) or the content expert (head instructor).
One particular aspect revealed through the interviews is connected to
the question of instructor and facilitator roles. VisArt was, as mentioned,
a collaborative telelearning scenario, where the students were to
collaborate in constructing a visual artefact for teaching a subject of choice.
In addition, however, it was also part of the DoCTA research project.
The instructors and facilitators all had roles as researchers, collecting
data about the students’ activities, a role supplementary to being an
instructor. When asking questions about their activities in the scenario in
an interview, it was sometimes a challenge to extrapolate the answers as
being an “instructor” or a “researcher” answer.
In addition to the instructors’ dual roles, there were a number of graduate
students also following the student groups, collecting data, and observing
their collaboration in TW. These graduate students had no formal role as
instructors or facilitators, but a preliminary glance at the data indicates
that they may have, in practice, been perceived and functioned as more
“experienced learners” for the students. The students asked them
questions via the Chat and Page tools, if they noticed they were online.
This may have affected the outcome of the scenario, in that their presence
204
Chapter 7 VisArt
may have affected the students’ work, and was not a planned part of the
VisArt activity. The implications of this, however, are not considered here.
Data collection
The semi-structured interviews were held after the completion of the scenario and lasted anywhere from thirty minutes, to one hour and thirty minutes.
The instructors and facilitators from University of Bergen were interviewed
face-to-face. The instructors from HSH and HiNT were interviewed by telephone. HiNT had, as pointed out, two persons officiating as instructors, and
they were interviewed together as a group. All the interviews were transcribed
in full length, and their contents are being evaluated.
The mail sent from students to the help/ training facilitator during the
scenario, amounted to approximately 80. There was also email sent
directly to the head instructor (i.e., not through the help/training facilitator).
All the email will be treated with respect to content, numbers, and time
of occurrence in the scenario.
Preliminary data analysis
Activity theory has influenced the data gathering and analysis in this study.
In this section, a preliminary evaluation is presented with examples
extracted from the data.
An Activity System
Engestrøm (1987)’s system for the analysis of activity, in relation to the
collected data, is presented in figure 58. The basic components of the
activity are the subject, the object and the community. Their relations
are, in principle, mediated. Tools, both physical and psychological mediate
the relation between the subject and the object. A division of labour
mediates the relation between the object, or what is being worked on,
and the community. The relation between the subject and the community is mediated by sociocultural rules.
The example of mediating artefacts given in this particular activity system is by no means extensive, and depends on level of analysis. This
level of analysis is an overview of VisArt as a whole, from the instructors’
point of view. The subjects in this particular focus on the VisArt scenario,
205
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
are the instructors and facilitators, and the object is the VisArt scenario,
which they want to transform. Tools that mediate that relation are for
example TW and e-mail, but also less “visible” tools such as language
(written and spoken). Sociocultural rules mediate the relation between
the subject and the community, and rules here can be for example
“professional” culture. The community is the social “surroundings”, or
environment of the subject, and can be the teaching institution, the funding organisation, and the group of instructors/facilitators. The objectcommunity relation is mediated by division of labour, and this means how
the instructors/facilitators decide to divide the workload among themselves.
Administering TW, creating the help system, or giving out assignments
are examples. The outcome of the activity, is the completion of VisArt.
Figure 58
VisArt instructors and facilitators as an activity system
Division of labour
The division of labour, which corresponds to the division of responsibilities
between the instructors, was very much planned in advance. This is
illustrated by one instructor’s response:
206
Chapter 7 VisArt
“So it was the help/training facilitator who had the overall
responsibility for the contact with the students, and he was supposed
to separate questions from technical ones to questions about content. The technical facilitator was technically responsible, and I was
responsible for content. The help/training facilitator catalogued
the questions. I was surprised not to receive more mail, but he took
responsibility... If he couldn’t answer, he passed them on.”
In this excerpt, the head instructor is explaining how they had planned
on helping the students during the scenario. A help-system, mediated by
email, was designed. The students were to send questions to the help/
training facilitator, who was to read them, and pass them on to the
assistant (head instructor or technical facilitator) who was assigned
responsibility for the area in question. As it turned out, he answered
most of the questions himself. One focus that will be examined in more
detail in my dissertation, is how the division of labour was in reality in
relation to how it was planned in advance.
Artefacts
The artefact that mediated the communication between the students and
the instructors/facilitators, was mainly email. Communication in TW also
occurred, both between instructors, and between students and instructors,
if the instructors happened to be logged on simultaneously with the
students. Between the instructors themselves, Email also seemed to be
the dominant vehicle of communication between instructors, although
the telephone was also used.
“In order to clarify problems related to running of the TW program,
talking to the technically responsible person, I used e-mail. And I have
also spoken to “Stord instructor” on the telephone... Here, it was
mostly face to face. I guess these are the three forms I have used.”
For the HiNT instructors who were physically located at the same place,
face-to-face communication in the form of scheduled meetings or
occasional meetings in the “corridor” was also mentioned
Sociocultural rules
Naturally, it isn’t easy to pinpoint culture or tacit, intra-group norms or
rules when oneself is an immersed part of that group or culture. The
207
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
instructors, however, seemed to have a mutual understanding of the
required work and the division of labour. Tasks that were assigned to a
particular instructor, were this person’s sole responsibility:
“Being part of it, it is hard to say. But you have your own tasks. Or
areas of responsibility, rather. … If I was withheld from doing it, it
was my responsibility to ask someone else to fill in.”
One might label this “professional”, in the broad, cultural, understanding
of the word.
Goals and motives
Understanding the goals and motives of the instructors and the objective
of scenario itself is revealed in the data. After analysing the interview
data, it became clear that the instructors had a personal interest in the
groupware technology. For example, one of the says:
“I suppose it has been a genuine interest in groupware, or collaboration
technology. (...)What I found particularly interesting in this tool is the
fact that the people are very much aware of each other, bearing in
mind the ”whiteboard”, where they made drawings together and
worked simultaneously. (...) The fact that the tool had many functions
making people aware of each other’s presence was very interesting
in itself, because this is new to me. I also participated in IDEELS...”
Other goals arise from an interest in collaborative learning, and in trying this
out in practice through the deployment of VisArt. The objective for VisArt
was, as given in the assignment, to give the students practical experience in
collaborative learning, mediated by an Internet tool, and to theoretically reflect
on the experience. There was a divergence of perceptions with the instructors.
The divergence can be described as a difference in opinion of the importance
of the product that the students were to produce — the visual artefact. Some
perceived this as secondary to the main goal, while others were preoccupied
with the quality of this artefact more than the collaboration process.
Issues in telelearning scenarios: organisational,
pedagogical and ethical
In the following section, a number of specific issues arising in the data
are discussed. These will be expounded further in the dissertation in
preparation (Wake, in preparation).
208
Chapter 7 VisArt
Virtual versus Real Environments. Being an instructor, or a student,
in a virtual learning such as TW, will differ from a “real” learning environment, in the classical sense, with regards to communication and learning
to be mediated by tools (mental or physical). This may affect the student—instructor relationship, the student—student relationship, and
certainly the student/instructor—tool/technology relationship. These
relationships require specific considerations in the planning of the learning
activity.
Focus on Collaboration Process. Although there was a general
satisfaction with the completion of the scenario, the focus was on tool use
and the collaboration process, rather than the end product. In addition,
the objective set for the students was to gain experience in a collaborative
telelearning activity, rather than producing a visual artefact for teaching
a subject of choice.
Scenario Planning. As one interviewed instructor pointed out, time
used in planning the telelearning activity such as VisArt exceeds, by far,
that of a learning activity where information and communication technology (ICT) is not a mediating instrument.
Resource Demanding. The instructor interviews revealed that VisArt
has been much more demanding on resources both in time spent as
instructors and in facilitating the technological environment for the students,
as compared to a non-telelearning course. This is contrary to the popular
perception of technology as “self-running”.
Instructors as Researchers. The VisArt instructors also had interest
in the activity as researchers. Some of the instructors reported in the
interviews that they felt themselves “drawn” into the activity. For example,
most of the instructors ran TW for the duration of the working day, and
spent a number of hours online every day,. While online they observed
the activity of the students, and were available for questions and tutoring. In addition, they commented that they did not only receive email or
requests for advice during regular working hours, but also in the evenings
and on the weekends. This is not usually the case in an ordinary semester
course. This “24-hour demand” for availability can put an amount of
stress on them if they are not careful.
Restarting the TW Server. Although it has been pointed out that TW
is not a particularly difficult groupware system to administer (see section
7.6), the server stalled during the beginning of the scenario. This was
209
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
due to an overload on the system’s buffer file — a result of an extensive
workload. In order to avoid a server crash, three daily restarts were
required. As the students worked at hours that suited them best (not nine
to four), including during the weekends, the TW server needed to be
operative at all times. Thus a schedule for restarting the TW server was
done planned and either a VisArt graduate student or the head instructor
restarted the server three times a day.
Importance of Facilitators. One instructor stated that the assistants/
facilitators, and the technical support, was far more important than the
instructors in a telelearning scenario. Perhaps this can indicate that it is
important to have a certain number of assistants, rather than a large
group of instructors.
Relationships. The instructor—student relation, largely mediated TW
and e-mail, was affected by the distributed nature of the learning situation.
It would seem, from the data gathered for this study, that the personal
contact with the students that follows from a classroom situation, or being
in the same room, where it is possible to see facial expressions and hear
the tone of a voice, is obviously weakened by collaborating and
communicating through an Internet tool. It is difficult for the instructor to
monitor the internal collaboration process in the group and to appreciate
their level of activity (both as a group and as individual students). This
makes it more difficult to give personal feedback, to help the students
whose activity level perhaps is below expected, and to lever group progress. In some respect they only saw the artefacts produced by the group
in the TW rooms (i.e, the product of the collaboration process) rather
than the collaboration in process per se. They did not, for example, have
access to the intra-group mail communication. Having access to it might
have helped their understanding of the intra-group collaboration process,
but it would also increase the workload on the instructors, and would also
raise certain ethical questions.
Standing and Staring in Silence. Some of the instructors expressed
hesitation in entering the group rooms in order to observe the activity of
the students. They were afraid that it would interfere with the group
collaboration process. One instructor, in an interview, rather humorously
pointed out that one could compare entering a group room in TW while
students collaborated, with going into a room where a small group of
people are working without knocking on the door, and just standing there
not saying anything, just staring. One can speculate that these feelings
210
Chapter 7 VisArt
can be to the strong metaphorical concept of rooms adopted in TW —
that they are perceived as someone’s personal space, and one is hesitant
to intrude.
7.15 Supporting Collaborative Telelearning
Research using Server Logs
ØYVIND MEISTAD
The focus of the Master’s thesis described here44 is on how automatically
generated log-files can contribute to research in understanding
collaboration patterns (Meistad, in preparation) in telelearning
environments. One challenge for research within collaborative telelearning
is what and how to collect and analyse data in a distributed environment.
Traditional qualitative methods such as structured and unstructured
interviews, participant observations, and video recording need to be
tailored to distributed participants. Once the obstacles in collecting this
type of data are overcome, questions about how to interpret and analyse
the data persist. Collection of electronically logged data is on the other
hand relatively straightforward. As Garton et al. (1999) point out, the
restrictions are often “the amount of server storage space and the
integrity of researchers and programmers in their study design (p. 92)”
and electronic data gathering “replaces issues of accuracy and reliability
with issues of data management, interpretation, and privacy (p. 92)”.
Generally, the amount of electronically logged data can easily be
overwhelming and again, questions of interpretation and analysis pose
real problems.
There exists a long tradition of using logged data in various usability
studies focusing on identifying events from log files such as the number of
errors occurring or when a menu is selected with the mouse etc., and
graphically visualising the results (e.g., Fjeld et al. 1998, Guzdial 1993,
Okada & Asahi 1999). Another interesting approach is to use the logged
data in conjunction with video analysis to effectively retrieve parts of a
video recording (Badre et al. 1995). Common in these types of studies is
that the focus is on the interaction between human and computer. Another,
44
This section is based on Meistad & Wasson (submitted).
211
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
somewhat different tradition is to use logged data to measure the usage
of various Internet resources such as FTP-servers, web-servers, etc. This
kind of information is likely to be most interesting for the Internet providers
providing the resources, but is also starting to gain attention from
researchers (e. g. Newhagen & Rafaeli 1997, Garton et al. 1999). While
most usability studies focus on the interaction between human and computer, the focus of the last kind is on the human communicating with a
remote computer, mediating his or her communication through his or
her local computer.
The tradition of using logged data for identifying patterns of collaboration
mediated by computers between two or more geographically dispersed
persons appears to be non-existent as no references to others with the
same or a related approach have been found. This study explores this
possibility and tries out various techniques, with a mere goal of examining
whether or not the approach is promising.
Method
The research was conducted using automatically generated logged data,
generated by TeamWave Workplace (TW) during the VisArt scenario. TW
comes with the usual logging mechanisms such as recording that a user
logs in and out of the server, moves between rooms etc., as well as a nontraditional feature, a repository. All rooms are saved in the repository, and
it thus becomes possible to recreate earlier states of any given room using
an option called version control. Both these mechanisms (the server log
and version control) are used in conjunction with two prototypical tools
developed by Mesitad (in preparation) and described below.
The TeamWave Workplace Server File
All major interactions with the TW server are electronically logged in a
server log-file called server.log. The file is automatically generated by the
TW server on its first start-up, and is then updated as the various users
interact with the server. During the VisArt scenario the file grew from 0
KB to over 2.2 MB with 38,207 lines. Figure 59 shows some extractions
from the VisArt server.log file. The extractions describe some of the author’s
actions when he logged into the VisArt server for the first time.
212
Chapter 7 VisArt
Fri Feb 19 15:25:08 1999 oyvind entered Overview-room
Fri Feb 19 15:25:26 1999 oyvind left Overview-room
Fri Feb 19 15:25:26 1999 oyvind entered Klasse-diagram
Fri Feb 19 16:32:35 1999 oyvind left Klasse-diagram
Fri Feb 19 16:32:35 1999 connection closed for oyvind
Figure 59
Extractions from the VisArt server.log file
As one can see in figure 59, each line starts with information about date
and time. Thereafter, the user is identified and the user’s particular action is recorded. For example, the second entry tells that on Friday,
February 19th 1999, at 15:25:26, oyvind left the Overview-room. In
addition to logging information about users entering and leaving rooms
(as shown in figure 59), the TW server logs users logging in and out of
TW, information about the server starting up, the creation of rooms, the
writing of rooms, and the server shutting down.
Although the structure of the server.log file is rather simple, the often
large number of users involved (e.g. in VisArt: 32 students, 3 instructors
and 10 researchers) creates such a large file that it is difficult to glean
information from the file. Therefore, a software tool that interprets the
entries has been developed.
The TeamWave Workplace Version Control
Every room created in TW resides on the TW server, and is stored in a
repository. When a TW user selects the menu option to save the current
room, or leaves a room when no other person is visiting the room, the
room is saved to the repository. Every tool created in each room is also
stored in the repository. As a room is saved, tools that have been changed,
or added, or moved around, etc. since the previously saved version of
the room, are saved. When a user enters a room, the last version of the
room and the last version of the tools residing in the room is automatically
retrieved from the repository.
TW’s version control enables earlier versions of the current room, or
earlier versions of a specific tool to be retrieved from the repository. The
average number of versions stored in the repository for a single room at
213
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
the end of VisArt was approximately 50. The version control option
enables a researcher to recreate the state of various rooms and tools
during one’s analysis. Thus, long after the completion of the scenario,
one is still able to follow how the scenario evolved for a particular team.
The potentially large number of available versions of one room or tool,
does, however, make such an analysis difficult. Therefore a software
tool, TRA, was developed to help researchers keeping track of how the
various rooms evolved over time.
Two tools for analysing log-files
Currently there is a lack of software designed for research on automatically
generated logs with the intention of identifying collaboration patterns. To
support the evaluations of scenarios where TW was used, it was necessary
to develop two prototypical tools for the analysis. The tools, the Server.log
analyzer (SLA), and the TWWRoom Analyzer (TRA), were both developed
in Java 1.1 using the Java Foundation Classes 1.03 as graphical user
interface components (when available in VisualAge for Java, the tools
will be modified to use the generally available Java Foundation Classes
1.10). The programs differ from each other in that SLA reads in one
log-file, sorts out a researcher specified group of persons, and then uses
the information to calculate some results from the log. TRA, on the other
hand, is a tool to help researchers in classifying the use of various tools
inside TW, organising their classifications, and registering how the use of
the tools evolves over time. In this study TRA was used in conjunction
with the TW version control and the output from SLA.
Server log analyzer
SLA takes as its starting point the server.log file. The VisArt scenario
was completed by 11 teams and SLA focuses on identifying the times at
which a team could have possibly collaborated synchronously in TW (i.e.,
two or more of the team’s members must have been logged on at the
same time).
To find out what types of collaboration was possible for a given team at
a given time, SLA reads through the server.log using a researcherspecified names of the team members. SLA creates a log for each group
member, as well as a group log consisting of every entry from the individual logs, appearing in the same order as found in the server.log. The
214
Chapter 7 VisArt
resulting logs (4 in this case) contain redundant information. SLA removes
unnecessary information from the group log, and adds some statistics to
the group log. An excerpt from such a log, (usernames and room names
have been changed) is shown in figure 60.
————————————————
- Wed Mar 17 , 1999 ————————————————
10:24:43 - Subject02 entered Teamx
10:28:27 - Subject02 left Teamx after 3 minute(s) and 44 second(s).
10:28:27 - connection closed for Subject02
*** subject02 - time in TeamWave current session: 4 minutes.
*** subject02 - time in TeamWave all sessions: 12 hour(s) and 31 minutes.
12:23:49 - Subject02 entered Teamx
12:31:39 - Subject02 left Teamx after 7 minute(s) and 50 second(s).
12:31:39 - connection closed for Subject02
*** subject02 - time in TeamWave current session: 8 minutes.
*** subject02 - time in TeamWave all sessions: 12 hour(s) and 39 minutes.
12:58:30 - Subject02 entered Teamx
13:02:05 - Subject03 entered Teamx
14:01:48 - Subject01 entered Teamx
14:24:31 - Subject03 left Teamx after 82 minute(s) and 26 second(s).
14:24:31 - Subject03 entered TeamRoomX
14:25:15 - Subject03 left TeamRoomX after 0 minute(s) and 44 second(s).
Figure 60
An excerpt from SLA’s group log-file
This log is used to generate a graph that illustrates the times at which the
various group members were logged in to TW. If the graph shows that
two or more group members were logged in at the same time, they had
the possibility to collaborate synchronously. Figure 61 shows a graph for
one of the groups from VisArt. The left column of the graph denotes the
time of the day. Each day starts on the bottom at 0, and goes up past 23
and up to 0 again. The bottom axis specifies the date that is displayed. If
a date is missing (e. g. Feb 26 and Feb 27), it means that none of the
group members logged in that day.
215
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 61
Graphs of Team Activity in VisArt
Interesting information may be found looking at the graphs. First they
tell us that from February 25 - March 24, there were only three times
when it was possible for the whole group to collaborate synchronously.
VisArt started on February 25 when the group members were told to
download the TW client, and log in to the server. They were given a few
days to carry out this task. March 1 - 7 was a week of training in use of
TW and collaboration, and March 8 - 26 was the period of carrying out
the design task. By examining the graphs, it becomes apparent that
subject02 did not log in to TW before the fourth day of the training
period. Is also becomes apparent that the formerly active subject01, did
not log in once between March 4 and March 15. Consequently, even
asynchronous collaboration is highly unlikely during this period. To explain
why these more or less negative patterns occurred, is difficult using a
simple log-file as the only kind of data. It is important, however, that the
researcher is aware that one of the three team members disappeared
for 10 days in the middle of a 4 week collaborative learning task. This is
valuable information that enables the situation to be investigated more
closely using, for example, interviews, and / or questionnaires.
216
Chapter 7 VisArt
Server log analyser, version 2
SLA is a fairly static application, and the type of output it provides is
limited compared to the possibilities. A new, more flexible and dynamic
version will be implemented based on the outcome of this project. Using
SLA in conjunction with TRA in an ongoing analysis, however, identified
a few urgent needs, thus a few version 2 functions have been implemented
in order to give the desired output from SLA. Due to limitations in the
graphical user interface (GUI) of SLA, the GUI of SLA version 2 (SLA2)
will be completely rewritten. Thus, the tool currently exists in two versions,
of which both have been used in conjunction with TRA.
The most important new function is the ability to sort the server.log based
on the rooms involved. SLA focuses on groups of persons, and filters
away all other information present in the server.log. For example, information that a researcher is present in the room at the same time as the
members of a team would be discarded. This is, however, important
information needed by TRA. Therefore SLA2 is implemented in such a
way that SLA2 does not filter away such information.
TWW Room Analyser
TWW Room Analyser (TRA) is designed to be used in an analysis of
collaborative learning environments that use a room metaphor or another
analogous structure such as a web page. In this study, TRA helps
researchers in classifying the use of various tools inside TW, organising
their classifications, and registering how the use of the tools evolves over
time. TRA, see figure 62, requires the researcher to use the TW version
control and the output of both SLA and SLA2 to obtain input data for
TRA. Ideally it should be possible to retrieve most of the information
needed by TRA from the TW repository, but that path has not yet been
investigated. Even then, some of the information needed by TRA, such
as a classification of the uses of the tools, would still have to be identified
by a researcher. Once the data has been input to TRA, the researcher
can use TRA to track changes to rooms and tools over time.
217
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Figure 62
One of many possible views of TRA
TRA, shown in figure 62, is a dynamic tool that is used to describe the
learning environment of a VisArt team. TRA comprises several graphical
user interface components that are displayed at different times. One
important component is a tabbed pane that helps the researcher quickly
navigate between various room states at different levels of detail. In figure
62, three tabbed panes are visible: the room tabbed pane where the
room “Team05” is active; the version tabbed pane where version “5” is
active; and the tools tabbed pane where the “Add new tool” tab is active.
Table 27 summaries the main components of TRA telling when each
component is visible, and describing each component’s input and output. In the table, one cell denotes one of the main components of TRA.
Table 27 should therefore be studied in conjunction with figure 62.
218
Chapter 7 VisArt
Table 27 The main components of TRA
Persons influencing the LE
LE include the following rooms
- Always visible
- Always visible
- List of users influencing the current team’s LE - List of all rooms used by the current team
- Possible to add new persons
- Possible to add new rooms
Tabbed pane (no 1) is always visible. A tab is created for each room from the room list above
Properties tab is always visible, and when active enables several attributes (e. g. where to save
the current analysis, time between each automatic save, whether to backup analysis) to be set
Room tab is always visible, and when activated enables several attributes (such as who
created and when created, etc.) of the current room to be set. The available tools and
available versions list become visible.
Available tools:
- Visible when a room tab is activated
- List tools available in the current room
- Possible to add new tools
Available versions:
- Visible when a room tab is activated
- Lists versions of the current room
- Possible to add new versions
Tabbed pane (no 2): is visible only when one of the room tabs is activated in the tabbed pane
(no 1)
Summary tab when active displays information about the state of the room (e. g. number of
tools, type of tools, if a tool is used asynchronously or not, etc.)
Version tab when activated displays information about the version (e. g. when created by whom,
extractions from log-files concerning the version, researcher comments to version, etc.)
Tabbed pane (no 3) is visible only when a version tab (from tabbed pane (no 2)) is activated
Add new tool tab when activated enables tools found in the current version of the room to
be added to the list of tools and a tool tab is created for the tool. If a tool existed in the
previous version of a room, it is possible to remove it from the current room. The researcher
can also copy in parts of log-files concerning the version, and write down some general
comments concerning the version
Tool tab when activated displays information about the tool type (e. g. doorway, postIt), who
created it, if it was used synchronously, asynchronously, etc. In addition, each tool can be
classified by a researcher according to the type actor (e. g. leaner, tutor, manager); the roles
taken; the modulator of the actor (i. e. individual, team, or entire group) etc. according to Téleuniversité’s model for a virtual campus environment (Paquette et. al., 1995). It is also left space
for other classifications, as well as general comments about the use of the particular tool.
TRA can present information about any number of rooms. For each
room there may be any number of room versions, with each room version containing any number of tools. As TRA is currently being used during analyses of VisArt, it is possible new features may be required. If so,
then TRA will be revamped and the user interface may change. The
219
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
TRA feature where the designer can record the use of the tools in a
particular room is shown in figure 63 (see the bottom pane with the
Designer tag and window for explaining tool use.
Figure 63
Using TRA to record analysis of tool use
Conclusion
The dissertation work described here has resulted in the development of
two prototypical tools that take automatically generated data as their
input. SLA does so as a fairly automated process, whereas TRA depends
on a great deal of interaction with the researcher. Information produced
by SLA is currently useful for identifying the times at which a team had
the potential to collaborate asynchronously or synchronously, as well as
well as if anybody else was logged in to the same room as one or more of
the team members. To say that it is possible to collaborate is, however,
220
Chapter 7 VisArt
not to say that collaboration occurred. Thus, SLA’s output can be used
to focus other methods that are being used to interpret the interaction
between users of TW through its various tools.
TRA was developed as an aid for studying the learning environment of a
team (e.g. the rooms involved, the persons influencing the environment,
the available tools, etc.) Incorporating Télé-universite’s tool classification
model for a virtual campus environment (Paquette et. al., 1995) in TRA,
it suggests how the particular uses of the tools found in the team’s rooms
may be classified. Using these tools in conjunction with other data gathering methods such as interviews or video observation, the researcher
will be able to develop a richer picture of collaboration patterns.
7.16 Coordination in collaborative telelearning
RUNE BAGGETUN
The challenge of collaboration between geographically separated team
members in a learning context is the focus of this section. In VisArt
participants collaborated using TW as well as a personal email
application. VisArt was studied in order to find out how the participants
coordinated their work using this suite of tools. Wasson (1998) identified
a set of interdependencies in collaborative telelearning (see table 4) and
the VisArt scenario provided the opportunity to study coordination in a
situation were the participants are faced with these (inter)dependencies.
Building on the concept of coordination mechanism (Schmidt & Simone,
1996) we decided to focus our research on the artefacts and procedures45
used for coordinating distributed work. A coordination mechanism is
defined Schmidt & Simone define as
“a mechanism that, by means of a set of conventions and
prescribed procedures and supported by a symbolic artefact
with a standardised format, stipulates and mediates the
coordination of the distributed activities of collaborating
ensembles”
45
A procedure is seen as a resource an individual can turn to, rather than a strict rule.
221
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Method
Data from 2 teams46 was gathered and studied. Some of the other
researchers also provided relevant data from other teams that they were
studying. In order to gain a deeper understanding of the issues being
explored, the activities of 6 students (2 teams of 3) were closely followed
using a number of techniques.
Virtual Field Study. The distribution of the participants gave little
opportunity to conduct a normal field study, but the features of TW gave
instead the opportunity to conduct a “virtual” field study. When logged
on I could observe the actions of team member inside the virtual world
(i.e., the TW rooms), as well as track any changes made by others since
the last time they logged on. What could not be observed was what actions
were taken outside TW. For example, it was impossible to know if the
team member from HSH surfed on the web while talking, in TW, to his/
her team companions.
Interviews. Three types of interviews were conducted. Three of the
interviews were conducted as “In-Depth, Open Ended Interviews (Patton,
1987)”. The second type was informal open-ended interviews. For
example, talking casually to team members in the hallway or at lunch.
Several times a participant who had some opinions to share or something
else that they wanted to tell approached me. The last type of interviews
conducted were “context interviews” conducted in TW using the Chat
tool. They are being referred to as “context interviews” because the
interviews were conducted with team members while they were working
(i.e., in context). All the context interviews were informal and were saved
as a text file using a special feature of the Chat tool in TW.
Automatic Data Logs. One important feature in TW is the automatic
logging of data. Almost all user actions are logged. The log analyser
tools SRA and TRA, as described in section 7.15, were used in this study.
Using these tools it was possible to see each team member’s movements
inside TW (e.g., from room to room), to see how long each team member
was logged on, and to see who was logged on at the same time in a
room. This gave an indication of the synchronous/asynchronous nature
of the collaboration within a team.
46
Rune was assigned 2 teams to follow.
222
Chapter 7 VisArt
Lotus Screencam. Lotus Screencam enables a researcher to record
what is happening inside a TW room without the researcher having to be
present. Screencam records movies of screen activity through a full video capture that includes all mouse-pointer movements and other screen
events. Screencam was used to record screen activity in several TW rooms.
These recordings are a rich source of data that can be studied over and
over again to find new subtleties in the on-line interaction.
Analysis
The concept of coordination mechanism can be used to determine how
the group invented, used and adopted mechanisms that supported the
coordination work during the scenario. A preliminary analysis has thus
far, revealed three collaboration patterns. These are:
• No initial discussion regarding the use of tools
• What was obvious for some was not obvious for others
• Different expectations regarding whole sequences of related actions
Each of these is briefly presented below from a coordination mechanism
perspective.
No initial discussion
There was no agreement or discussion of how to use the different tools
before actually using them. This led to some unfortunate implications. For
example, the team members on one team used different tools for the
same purpose and time was spent checking the different tools. For example,
group messages were left on several different tools in several different
rooms instead of only on an agreed upon tool in one room. From a
coordination mechanism perspective we can see that although an artefact
exists (e.g., a message board), it is not very useful unless an agreed upon
procedure for use, i.e., a coordination mechanism, is adopted.
What was obvious for one was not obvious for others
With procedures missing and no conventions developed, the use of tools was
dependent upon the team member’s own skill and ideas about when and
how to use the different tools. This again led to some unfortunate implications.
Some team members put a lot of effort into using a message board for
communicating messages to the team while another team member used the
223
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
message board just to get information or post an answer. In addition, a
completely different tool, i.e., the PostIt note, was used to communicate
messages, and it placed in a different room (in this case it was placed in the
individual’s room and not the team room). What was obvious for some was
not as obvious for all the team members. In this case the whole team should
have used the one message board in the team room as intended by the one
who created it. Again we see that the missing procedure for how to use the
tool created trouble. The procedure developed was, in this case, personal.
It seemed that each team member developed personal “styles” of
coordinating their own effort to accommodate the others way of working.
Different expectations
Different expectations regarding whole sequences of related actions were
different among the team members. Because of this simple action such as
dividing labour or discussing topics was not as efficient as it could have been.
For example, in a chat session a question or suggestion from a team member
would often get no response because the others thought that being silent
was the same as saying “I agree”. The one with the suggestion or question
was, however, accustomed to getting feedback. From her experiences in
Internet chat groups she expected at least to get a smiley (or some other
feedback) otherwise she would feel ignored or that her suggestion was a bad
one that the other did not want to discuss. The no feedback strategy in a
virtual environment means no information at all because the lack of other
cues such as facial expressions and body movements are missing. We see
(again) that missing procedures are causing problems.
Summary
Even though there were procedures missing for how to use tools, both
teams collaborated well. Using mostly the chat facility in TW they agreed
both on what to do when they were online, and what to do before the
next meeting. The more interesting findings in this study that will be
elaborated further in my dissertation (Baggetun, in preparation) were:
• Personal styles were developed when there was a lack of conventions
• We need to determine how to help users develop good procedures
and detecting bad ones
• We need to find out how to help the students maintain a balance
between personal styles and agreed upon procedures.
224
Chapter 7 VisArt
7.17 Facilitating Help and Training in VisArt
HELGE UNDERHAUG
The main focus of this Master’s thesis is the design and deployment of
and assistance in VisArt. The students began VisArt with a one-week
training period that provided training in both use of the TW tools and in
collaboration. Assistance was provided from the start to the end of VisArt.
An evaluation study of the training and assistance asks the following two
questions:
How effective was the training and assistance designed for learning
to use the TW tools and learning to collaborate?
How can the training and assistance be improved?
Section 7.7 described the training design and introduced the training
assignments. The organisation of help and assistance was described in
section 7.5. This section describes the data collection and presents a
very preliminary analysis of some of the data.
Data collection
Several data collection techniques were used, including:
•
•
•
•
•
email that was sent to the head assistant
email that was sent within 2 groups
interviews with two members from one of the groups
notes taken while observing collaboration/group work within TW
questions regarding training and assistance included in the pre- and
post-questionnaires
• a web-based self evaluation test
• counters on the TW web help pages
• saved chat’s
The email that was sent to the head assistant was collected in order to
analyse what difficulties the students experienced during both the training and the design activity. The team email collected from two teams
will be used to determine what they communicated about outside TW,
and if and how they asked for help from their team members.
225
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Notes taken while observing the 2 teams collaborating in TW should
contribute to understanding the other data. In addition, Chat’s from these
sessions have been saved. An interview with one of the teams was held at
the end the scenario to supplement the observation notes and the chat’s.
The self-evaluation test was developed to help the students determine
what they had learned through the training phase and to help identify if
they perhaps should do more training before starting the design activity.
This gives us data on how they felt their progress was during and after
the training phase. The pre- and post-questionnaires also contained
questions regarding the training — how they experienced the training
and asked if they had any comments on possible improvements.
Preliminary analysis
A very preliminary analysis reveals that the majority of the students had
some experience with collaboration / teamwork from both prior education
and work life. This must be taken into consideration when looking at
what the students learned about collaboration in the training phase. Figures
64 and 65 present background questions on collaboration from the profile-questionnaire that was filled out before the students started the training phase.
Figure 64
Experience with collaboration from prior education
A question about the usefulness of the training in TW given before the
design activity was included in the post-questionnaire. The question was
included in the post-questionnaire47 in order to give the students the
47
As opposed to the pre-questionnaire which was also given after the training but
before the design activity.
226
Chapter 7 VisArt
opportunity to finish the design task before judging if the training ha
been useful. Figure 66 shows that the majority of the students felt the
training was useful.
Figure 65
Figure 66
Experience with collaboration from work life
Usefulness of the training in TW, after completion of the
design task
The tendency so far seems to be that most students were rather pleased
with the training phase. Some comments on the post-questionnaire also
indicate that it could have been done better:
“There should have been better time on the training”
“It was a bit short, I got help from a fellow student. This made it
easier and more effective”.
Both students in these examples complain about the length of the training phase. This could be attributed to insufficient information about the
upcoming scenario that caused some students to be delayed in starting.
227
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Or, it could be attributed to the minor difficulties some had with getting
attached to the server. This last reason is also supported by the fact that
in the first week of the training, 24 emails requesting help with technical
matters were sent to the head assistant. Further analysis will try to
elucidate the dissatisfaction with the training.
228
Chapter 8 Results and conclusions
Part III
Results and
Conclusions
229
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
230
Chapter 8 Results and conclusions
8 Results and conclusions
This part of the report discusses the results of the first phase of project.
Two issues that have been central to the project are the identification of
collaboration patterns and challenges in studying students’ online learning.
These two issues are discussed below. Section 8.1 presents the very
preliminary results of our integrative analysis that aims to identify
collaboration patterns in collaborative telelearning environments. In
section 8.2 a brief, but important, discussion on methodological issues
for studying online environments is presented.
8.1
Collaboration Patterns
Collaboration patterns define sequences of interaction among members
of a team (such as students) that satisfy established criteria for collaborative
behaviour. We have already reported (see sections 2.2-2.4) on several
such collaborative behaviours: genuine interdependence, division of labour, sharing information, awareness of others, joint thinking, etc. Although
collaboration patterns can be found in physical as well as virtual
environments, in DoCTA we have focused on how collaboration patterns
manifest themselves in virtual environments.
Finding collaboration patterns
In the VisArt scenario we have searched for collaboration patterns by
analysing interaction data from data logs, videotapes, observations, and
interviews both between students and between students and facilitators
(instructors and assistants). We have identified several instances that we
believe can be characterised as collaboration patterns (Wasson & Mørch,
to appear). Four of them will be explained and discussed below. However,
231
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
the reader should keep in mind that our results thus far are preliminary48
and that we expect to report more on this later. We see great promise in
identifying collaboration patterns and expect them to be useful in further
work on virtual learning environments. After the four patters are described,
a collaboration pattern structure that is shared by all the collaboration
patterns we have identified is presented. The section concludes with a
short discussion future work.
Four collaboration patterns
Four collaboration patterns are described and discussed below. The
patterns have been identified in the VisArt scenario by analysis of the
data reported earlier. We use examples from the data to illustrate the
patterns by showing concrete, but representative, activities.
Pattern 1: Adaptation
Synopsis: This pattern describes how students gradually adapted to each
other’s practices when working together to solve a common problem.
There were no established protocols for how to use the different tools in
TW. The students used a wide variety of tools when communicating with
each other (e.g., a message board, shared rooms, chat, post-it notes,
etc.). Indeed, several of the tools could actually be used to accomplish
the same end and since the students preferred to use tools they felt
comfortable with they would choose differently. After an initial phase of
interaction, however, the more or less arbitrary tool use pattern changed.
Here is an excerpt from an interview that illustrates this change:
Sam: “… but also Pat had a tendency to use post-it notes inside his room”
Interviewer: “…Yes, they are really popular.”
Sam: “We had to run into his room and see what new post-it notes
had been posted there instead of looking at the message board or
the group room.”
In this excerpt one of the students (Sam) has adapted to another student’s
(Pat) way of communicating by placing post-it notes in his private room and
48
Remember that the majority of the data collected is part of a set of Master’s
theses that are in various stages of completion.
232
Chapter 8 Results and conclusions
waiting for people to come by and read them instead of using the shared
room (group room). The shared room had been explicitly designed to be a
common meeting area. In spite of this, the student (and later the others)
adapted to one student’s ways of using a communication tool and ignored
the initial set-up of the public room. The facilitator of the above scenario
(i.e., the observer who noticed it) summarises it in the following way:
“A person enters into his own room looking for messages. He
then takes a roundtrip through the other team member’s rooms
to see if any messages have been posted there. He then returns
to his own room”.
This adaptation to a new team behaviour that followed as a result and
persisted throughout the scenario we have identified as the adaptation
pattern. We call this a collaboration pattern because we have found
similar interaction sequences in the other teams as well.
Pattern 2: Coordinated desynchronisation
Synopsis: This pattern describes how coordination of activities between
team members changes after they have identified a common goal.
Many of the teams would start working synchronously, for example by having a simulated real-time meeting. In the meeting the team members would
agree upon a goal to pursue, divide the work and then work more or less
asynchronously to accomplish it. The latter kind of working meant fewer
same-time meetings but required regular “coordination points” to ensure
that the divided work would progress towards the common goal. TW tools
supported this kind of coordination —they served as coordination mechanisms.
Here is an excerpt from a session that started with a conventional (realtime) discussion of meeting times:
S1: I vote that we work individually and that we meet Monday
or Thursday morning.
S2: Tomorrow?
S1: Yes, we must meet before Monday at 5pm.
S3: I think tomorrow is too soon. I’ll go for Monday at 5pm.
S2: (…) Monday is better for me too.
S1: OK, Monday it is.
233
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
When a meeting time later had to be rearranged because one or more
of the students could not make it they would leave messages for each
other in their TW rooms, or they would send emails. The data from later
activities shows a gradual shift from synchronous to asynchronous
communication. This is further evidenced by a comment from S2,
expressed as a post-it note in TW:
“It’s really not necessary that we meet again until we have done
what we have agreed on Monday, is it? We don’t have to be online
at the same time to work on our rooms. We can look at what the
others have done while we are logged on, and leave a post-it, like
this one, if we want to communicate on anything. What do you
guys think?”
After having deciding upon the goal, the students would divide the work
among themselves, each being responsible for one part and work
asynchronously on this. A reason for this shift from synchronous to
asynchronous communication can, in part, be explained by the following
comment by the facilitator:
“They (the students) would work on their own — creating the
rooms — and the others would drop in and comment on the
work that had been done and proposing changes they thought
should be made.”
Not only were the students working on their own in an asynchronous
fashion; they were coordinating the activity by giving feedback to each
other.
We also see the Adaptation pattern at work in this example. The
asynchronous communication is carried out in a situation satisfying the
conditions of the Adaptation pattern as described above. This instance of
the Coordinated desynchronisation pattern uses the Adaptation pattern
as one of its parts, and we can say that the larger pattern is composed of
a smaller pattern. This part/whole structure can be found in many
examples and is similar to how patterns are organised in other “pattern
languages” (e.g., Alexander et al’s (1977) architectural patterns and
Gamma et al’s (1995) software design patterns).
234
Chapter 8 Results and conclusions
Pattern 3: Constructive commenting
Synopsis: This pattern describes commenting behaviour. Comments that
are neutral (e.g., just to the point) are perceived to be less useful than
comments that are also constructive (e.g., suggesting what to do next) or
supportive (e.g., encouraging).
This pattern is related to Pattern 2 and often used as one of its
components. We found that lack of feedback was perceived to be
problematic in several of the teams. Students wanted to get feedback
from each other on the artefacts they have made in their rooms. Feedback is also required to keep each other up to speed and moving in the
right direction towards the common goal. This could be accomplished by
using the tools such as a message board, or post-it notes, etc. A chat
dialog that took place between students illustrates this pattern.
While collaborating in TW to create a class diagram (see section 4), two
students were using the chat tool to make proposals to each other and
waiting for responses before continuing.
Initially – student B asks for permission to execute her proposals:
B: What use cases do we have?
A: What about “Making Appointment”?
B: I agree. But we can have two kinds of appointments: through
dentist and through assistant. Will this require two sub-cases?
A: Let’s assume that making appointments can only be done
through a secretary
B: OK
Later this interaction sequence (of getting a confirmation before continuing)
changed. One would post a diagram and keep on working until they
heard something from the other person. The following excerpt illustrates
this as well as the linguistic mechanisms used to coordinate the activity.
Later – both of them are commenting and encouraging each other:
A: I have changed the diagram. Do you agree?
B: Yes, it is great! You have created a connection between “report” and “dentist”
A: Yes
…
A: You impress me!
235
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The initial commenting is somewhat of a hindrance (time consuming)
since feedback is requested and given at every atomic step in the interaction. However, in the latter case the student A is given the opportunity
to change something that has built up over some time (a diagram) and
the changes are likely to have more impact compared to the previous
version of the diagram. The author of the changed diagram will therefore be subject to more severe commenting than in the one-step-onecomment case. When commenting of the latter kind is constructive and
encouraging it is perceived to be more supportive of collaborative
behaviour. Thus, we call this pattern Constructive commenting.
Pattern 4: Informal Language
Synopsis: This pattern describes how interaction often starts in a
formalistic style and gradually becomes more informal as team members
get to know each other. Frequent use of slang words or dialects local to
the community working together is common in instances of this pattern.
This dialog is extracted from a dialogue between 2 students (bruker1 and
bruker2) who are, as in the previous example, creating a class diagram
together. In this first part, the two are “getting to know” each other49 :
[12:52:40] bruker1 says: Skal jeg lage flere aktører? Tannlege osv?
(Shall I make more actors? Dentist, etc.?)
[12:53:15] bruker2 says: Greit (OK)
[12:54:45] bruker1 says: Enig i at vi har alle ansatte på høyresiden?
(Agree that we have all the employees on the
right?)
The next excerpt is taken from the dialog about half an hour later. The
two seem to feel more comfortable with each other. They both use slang
or “dialect” 50 . Also, they both use abbreviations and uncompleted
sentences:
49
50
In this excerpt and the next example, the original Norwegian dialect is given in
Italics with an English translation in parentheses.
In Norwegian there are 2 formal languages and an unknown number of dialects.
Every little place in Norway has their own dialect. The two students in this
example come from somewhere outside Bergen (on the western coast) and
somewhere close to Bodø (in the north) – their dialects are very different!
236
Chapter 8 Results and conclusions
[13:31:39] bruker1 says: Se! (Look!) (Subject points at new connectionarrow)
[13:31:48] bruker2 says: Kossen? (How?)
[13:31:51] bruker1 says: Vi har laga Inheritance-pil! (We made an
inheritance-relation!)
[13:32:14] bruker1 says: Den ligg nederst i “tabellen” av link-typer.
(It’s in the bottom of the “table” of link-types)
[13:32:21] bruker2 says: Aha! (Aha!)
[13:33:23] bruker1 says: Ka no då? (What now?)
[13:33:49] bruker2 says: Forbindelsar... (connections...)
[13:34:08] bruker2 says: Ml.klient - tannlege? (bet. Client- Dentist?)
[13:34:35] bruker1 says: Ja, selvfølgelig... E litt sein i hodet i dag...
(Yes, of course…My head works a bit slow
today…)
When the students had “warmed up” to each other, they were more
comfortable with each other. They would often shift to an informal
language and use slang words and local dialects during the interaction. In
the excerpt above one of the students used the word “Kossen” (How)
while communicating using the chat tool. This term is not common in
written Norwegian, but it is common in regional speech. By using the
word Kossen, the student (bruker2) identifies himself as someone coming
from a certain region in Norway (in this case Bodø). This makes the
interaction much more personal and informal.
Structure of collaboration patterns
We anticipate that a common structure for all the collaboration patterns
can be useful. First, it will simplify later reuse since patterns can be
stored in an indexed catalogue of patterns. This similar to how patterns
are organised in other pattern languages (e.g., architecture and software). Second, it becomes easier to search for new patterns when the
characteristics of existing patterns have already been established. We
tentatively propose that each pattern has the following eight
characteristics: (1) index, (2) name, (3) short description, (4) related
patterns, (5) example, (6) conditions, (7) mediating artefact(s), and (8)
references. Each of the items is briefly explained in Table 28 next page.
237
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Table 28 Characteristics of Collaboration Patterns
Characteristic
Description
Index
The list of patterns is enumerated to ease later retrieval
Name
Each pattern has a unique name (1-3 words)
Short description
A sentence or two giving the essence of the pattern
Related patterns
Patterns above and below in the organisational hierarchy of
patterns
Example
Shows an instance of the pattern (as in the previous section)
Condition(s)
In which situation(s) does the pattern apply
Mediating artefact(s)
What tool(s) are used (if any) when executing the pattern
References
Where has this pattern been published or reported (if at all)
As mentioned above, this is a tentative structure that will continue to
evolve as we carry on our analyses of the collected data.
How we plan to use collaboration patterns in further
work
In this section we have described our preliminary analyses through which
we have identified and structured four collaboration patterns. We consider this a first step towards a new and promising avenue for further
work. Some steps we plan to take in the future include:
• Document and organise the collaboration patterns other researchers
have reported51
• Find out what computational mechanisms in groupware tools are
supportive of / hindrance to collaboration patterns. This is the topic
for a DoCTA Masters thesis. Asking “How do the various collaboration
patterns put different demands on the design and use of
groupware?” the focus of Pedersen’s research (Pedersen, in preparation) is to analyse how the collaboration patterns identified above are
(or may be) supported by groupware applications. He will furthermore
investigate what communication modes should exist for collaborative
work to be effective.
51
Although they may not be using the term “collaboration pattern”.
238
Chapter 8 Results and conclusions
• Find out how electronic (intelligent) agents can identify incomplete
collaboration patterns and then advise the participants in the activity
of this discovery so that it can be completed (assuming the agent is
right) or discussed (assuming the agent is wrong). This is also the topic
of Silje Jondal’s Masters thesis.
• Find out whether collaboration patterns really are useful. This may
include interviewing “collaboration experts” (if they exist) and asking
them to interpret and analyse our patterns to see whether or not they
make any sense. However, it is most likely that collaboration patterns
emerge because of particular characteristics of the actors, their social
environment and the pedagogical and technological constraints so we
will need to carefully analyse our empirical data and try to find out
when which collaboration patterns worked for whom. This may enable
us to suggest “efficient” collaboration patterns for a given situation.
• What are the implications of these collaboration patterns on
pedagogical, technological and organisational design?
• How can collaboration patterns inform our training of students in
collaboration?
When we began discussing collaboration patterns among ourselves, several
questions arose. These questions still do not have answers (some thoughts
around the first one were given above). Two examples are:
How are collaboration patterns influenced by the student’s social
situation (whether they have family, work full time, etc.)?
How can we to distinguish between modes of interaction such as student/student, student/facilitator, and student-tool (HCI)?
In the first instance, it is most likely that external situations will put strains
on collaboration patterns, for example, causing some parts of a pattern
to remain uncompleted because the student has to work from 9-5. This
may create the need for modified (relaxed) patterns, credit for partial
fulfilment of a pattern, etc.
In the section instance, the patterns we have found are primarily for
student-student interaction. Student-tool interaction is included in the
mediating artefact characteristic of the collaboration pattern structure.
This characteristic may or may not be present. A more detailed look at
the data will enable us to identify which tools were involved. In addition,
a closer look at student-facilitator interactions is underway.
239
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
This preliminary work on collaboration patterns is, however, already
influencing the analyses of one of the Masters theses. In his preliminary
analysis, see section 7.13, Andreassen (in preparation) dealt with aspects
like co-ordination, communication mode, division of labour, and feedback and identified a few aspects of work organisation that occurred in
VisArt, as above. The emergence of an explicitly formulated common
goal seemed to reduce the need for same-time meetings, leading to a
shift in communication mode, from largely synchronous to asynchronous.
This shift was accomplished also through a process identified as coordinated desynchronisation. In addition, a more asynchronous form of
work organisation may be ascribed to the division of labour. This, and coordinated desynchronisation, may, but need not, lead to a co-operative
rather than a collaborative form of work (cf. Dillenbourg,, Baker, Blaye
& O’Malley, 1996. Each team member having his/her own area of
responsibility, without the final outcome being dependent on the results
of the work of the other team members, may be unfortunate in that it
reduces both the need and will to provide feedback. Feedback, leading
to reflection on the work and learning of both oneself and others, may be
seen as an integral part of a concept of collaborative learning, and settings
promoting collaborative work organisation should therefore be provided.
In the initial phases of a collaboration effort like this, a sort of double
communication, when more than one tool is used to inform other team
members about a, for example, a changed meeting time, might occur.
This sort of communication may be reduced or disappear with improved
technical understanding or changed work co-ordination over time, but
might be avoided with sufficient training and examples on how different
tools can be used for co-ordination purposes.
Finally, this work in DoCTA on identifying collaboration patterns is in its
infancy. We feel, however, that it is promising and that our continued
scrutiny of the collected data will most likely reveal additional patterns.
8.2
Studying students’ online learning
A major challenge for today’s researchers studying telelearning (or online)
learning environments is how to design their studies. A simple question
of what and how to collect and analyse data becomes a major obstacle.
240
Chapter 8 Results and conclusions
Even “traditional” ethnographic studies that collect qualitative data do
not readily suit these distributed ICT environments. As Steve Jones
explains in the Preface to the collection on Doing Internet Research:
Critical Issues and Methods
“It [the book] is a result of discussion with many scholars from a
wide variety of disciplines who believe, as do I, that simply applying
existing theories and methods to the study of Internet-related
phenomena is not a satisfactory way to build our knowledge of the
Internet as a social medium. Consequently, this is not a book that
will (at least not in any direct way) help people to use the Internet
as a research tool. Rather, its goal is to assist in the search for, and
critique of, methods with which we can study the Internet and the
social, political, economic, artistic, and communicative phenomena
occurring within, through, and in some cases, apart from but nevertheless related to, the Internet. (Jones, 1999, p. x).”
Methodological issues have been a central discussion point in project
DoCTA as we struggle to come to grips with what and how to evaluate
our scenarios. As the reader has seen (if one managed to read all the
studies presented in Part II J) we have carried out a wide variety of
evaluations. These evaluations complement one another and have
provided us with a multifaceted picture of the telelearning environments.
Interpretation of this picture has only just begun!
As the “field” of studying online learning environments is in its infancy,
there are no “off the shelf” methods and techniques to apply. We have
tried to be keenly aware of the limitations of our studies and the challenges
of adapting methods to these types of studies. This section reflects on
some of our data collection techniques and shares our experiences.
Methodological issues
During this project a major challenge has been to find and adopt a strategy
for examining students involved in collaborative telelearning. In this project
the strategy applied has included of a number of different techniques
and methods, ranging from online questionnaires, keeping a record of
and receiving copies of the students email, using the server logs generated
from the software used, and online observations and interviews.
241
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Online observations
This technique consisted of immersing oneself in the virtual environment
and following the collaboration in the actual medium. This means that
the researchers regularly followed the students’ activities in the virtual
environment and in this way tracked changes made in this environment.
Observing them when they were collaborating synchronously, and
attempting to follow the different paths of communication and the
collaboration process, was also an important aspect.
There were some considerations that the researchers had to make in
relation to these observations. When the students were logged on they
immediately became aware of the researchers’ presence in the environment, and often asked them questions regarding the organisation of the
scenario, problems they were facing or to clarify certain aspects of the
task they were given. Consequently, the researchers that were present,
in some cases, also took on the role of facilitator. A possible reason for
this might be attributed to the design of the actual groupware system
that was used (TeamWave Workplace). It is designed to support a
workspace awareness (Gutwin et. al., 1995) so that it is easy to know
who else is logged on and what they are actually doing and “where” they
are located within the virtual environment. In relation to these
considerations the observations should, at large, be classified as a certain
form of participant observations, emphasising that the researchers had
a special role in the scenarios and that the interventions by the researchers
most often were initiated by specific questions from the students.
Online interviews
Some of the researchers have carried out online interviews both using
the groupware system and email. The interviews that took place in TW
were rather unstructured and were done synchronously, while the email
interviews were asynchronous and fairly structured. Both methods
produced interesting data.
Collecting email and chats
A technique that also has been used is collection of emails and chat-logs.
The email was mainly gathered by setting up common team addresses
(one for each team) that included a researcher in the list. Another method
consisted of simply asking the students to save and forward the chat-logs
242
Chapter 8 Results and conclusions
(as these were not saved automatically) and email. As the different
researchers were collecting these kinds of data, they found it much easier
to arrange the collection of the data with an informant (i.e., one of the
participating students) they could meet face-to-face. An important lesson here is that the researcher has to be rather flexible when s/he is
looking for possible sources of information and be actively in contact with
the informants in order to gather the necessary empirical data and get a
rich picture of the online activities.
Server logs
The communication that was mediated through the groupware system
(TW) automatically leaves electronic traces in the server logs. These traces
can be exploited when doing an analysis of the collaborations in this
virtual environment. The data being logged, is not just statistical data
recording “who is logged on when”, but includes a periodic chronological
recording of all artefacts in the environment. This means that we can
recreate versions of the environment to study the use of artefacts over
time and the creation and development of the artefacts produced in the
collaboration process (Guribye & Wasson, 1999). Overall this is a very
valuable source of information when doing online research. The details
of how this information was dealt with are treated in section 7.14 of this
report and in Meistad & Wasson (1999).
Online and offline activities
As all of the techniques mentioned above are important when studying
students’ online learning, one should not assume that the students’ online
and offline experiences can be strictly divorced (Kendall, 1999). In fact
Jones (1999) argues that “to study it [on line] as if it were somehow apart
from the “off-line” world that brought it into being would be a gross
mistake…on-line experience is at all times tethered in some fashion to
off-line experience. (p. xii)”. The implications of this duality is that we
need to have an understanding of how their online experience fits into
their offline life. In DoCTA we strive to maintain this view, but it is
challenging. It is important, however, to address issues related to both
online and offline activities and how these are interrelated and mutually
influence each other in order to gain an understanding of the collaboration
process.
243
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
One implication of this is that the data collection should not, if at all
possible, be limited to the sorts of techniques described above, but one
should try to seek other sources of information as well. These might
include techniques such as face-to-face interviews, telephone interviews,
and offline observations (where the actual students are physically situated).
Data collection in DoCTA included all of these techniques.
244
Chapter 9 Summary and Conclusions
9 Summary and Conclusions
This section concludes the report. Section 9.1 gives a summary of the
issues we have addressed in the first phase of DoCTA. The report ends
with a reflection on how we have meet our objectives in section 9.2
9.1
Summary
DoCTA is a large project with many issues being addressed. This section
attempt to summarise some of the issues we have explored.
The theoretical foundations presented in section 2 have influenced the
design and evaluations of the IDEELS and VisArt scenarios. In particular,
Salomon’s (1993) work on genuine interdependence and Gutwin et al.’s
(1995) work on awareness affected the design of the collaboration
assignments for both IDEELS and VisArt. Furthermore, both of these
perspectives influenced the evaluation of TeamWave Workplace
presented in section 5. Coodination theory serves as the foundation for a
number of our studies on how students coordinate their work in these
online learning environments.
Section 3 presented the conceptual framework that has influenced our
evaluations. In particular Activity Theory has played a role in several studies.
Activity Theory provides a lens through which we can look at the world. It
is useful to orient thoughts and research questions. It provides a number of
methods/tools shaped by a general theoretical approach. It can be used
to guide methodological decisions regarding evaluation (how technology is
used) and to a lesser extent regarding design (how technology will be used).
Engeström’s (1987) triangle (i.e., activity system), employed in several
analyses (see sections 6.1, 7.13 & 7.14) provides concepts and ideas that
245
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
make it easier to capture and analyse the context of the activity. In
particular it is useful to draw focus towards the division of labour, use of
artefacts, sociocultural rules, community, the object of the activity, and the
goals and motives of the subject. By first understanding the activity system,
one is better able to focus an evaluation on particular aspects of the activity.
Guribye, in describing his evaluation study of IDEELS (Guribye, 1999;
Guribye & Wasson, submitted), found Leontiev’s (1978) distinction of the
hierarchical structure of an activity useful in revealing some problems that
occurred during the collaboration process (see section 6.1). Finally, Guribye argues that in order to gain a thorough understanding of the role of
technological artefacts (including ICT), it is not sufficient to see only the
technology or a tool in isolation. Rather, these artefacts must be seen as an
integral part of human activity and human activity is always a complex and
dynamic process that needs to be studied in situ. This can be done in a
structured and informed way by using AT as an analytical framework
The evaluations of the groupware systems presented in sections 4 and 5
can serve as a starting point for those interested in choosing a collaborative
technology for use in a learning environment. Higraff’s evaluation of 4
groupware systems (see section 4) concluded with a preliminary set of
suggestions for what groupware systems should contain. The thorough
review of TeamWave Workplace in section 5 is complemented with the
usability study of TeamWave Workplace presented in section 7.11 and
the study on coordination mechanisms presented in section 7.16.
Section 6 presented the IDEELS scenario and described two evaluation
studies. Guribye’s study, section 6.1, focused on intra-group collaboration
in a team of students participating in the IDEELS simulation game. His
study, in many ways, has influenced the design of the evaluation studies of
the VisArt scenario. Ekker (section 6.2) was interested in changes in attitudes
towards simulation-based distributed learning. He was able to draw two
conclusions. First, the software utilised (OPUSi) may be improved in order
to alleviate some of the frustration experienced. Second, in preparing for
future simulations it will be important to focus on integrating all members
of the team in order to avoid having one or two participants “taking control”.
VisArt is presented in detail in section 7. All phases from design and
deployment to evaluation have been documented. Our description of the
design process should be of interest to those preparing to design their
246
Chapter 9 Summary and Conclusions
first collaborative learning scenario. The numerous evaluation studies of
VisArt complement one another and it will be a challenge for us to
integrate the results when the individual anlyses are completed. A short
reflection on some of the preliminary observations, however, can be made.
For example, Wake’s evaluation of how the VisArt instructors and faciliators
organised their work (see section 7.14) draws out aspects of deployment
to be considered. Regarding practical experiences, the interviewed
instructors/facilitators all focused on the amount of time used in the planning
and running of a telelearning activity. They felt that the resources needed
were quite extensive. Unforeseen technical problems can occur, and there
is high demand of feedback made by the students via e-mail and other
electronic communication tools. A major motivation for many students
engaging in telelearning activities is that it is flexible, that is, they can work
at hours that suit them best. This puts pressure on the instructors as they
want to provide feedback as fast as possible, but they can not be accessible
around the clock. Emerging from these time demands is a need for several
assistants — for both technical and content aspects.
One desirable quality of groupware tools used in collaborative telelearning
activities, is tied to the extent the instructors can get feedback on the
student’s collaboration process in the activity. Wake’s study found that
the facilitators felt that their ability to follow, in TW, the students’ progress in the learning activity was difficult and thus their ability to give
feedback was limited. In ordinary classroom activity, the instructor can
follow the activity level of the students relatively easily. In an online environment where students and instructors are not necessarily in the environment at the same time (or in different places in the environment), one
is dependent on students’ self-reports, requests for assistance, and
inspection of the used and produced artefacts. This results in the instructor
having problems in, for example, giving adequate support to individual
students, or teams, that are struggling.
Although little analysis has been carried out in Underhaug’s (see section
7.17) evaluation of the training and assistance in VisArt, the tendency so far
seems to be that most students were rather pleased with the training phase.
Some comments on the post-questionnaire also indicate that it could have
been done better. Several of the comments point to the length of the training phase. As Underhaug points out this could be attributed to insufficient
information about the upcoming scenario that caused some students to be
delayed in starting. Or, it could be attributed to the minor difficulties some
247
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
had with getting attached to the server. Further analysis will try to elucidate
the dissatisfaction with the training and these results will be used to suggest
improvements that can be made in future training situations.
As we have discussed in section 8.2, studies of online learning environments
are challenging. We have tried to meet this challenge by focusing studies
from several perspectives. Even a more “traditional52 ” usability study by
Rysjedal (see section 7.11) focused on studying the technology in use thus
was challenged by how to collect the data to best answer questions about
efficiency, effectiveness and satisfaction. His preliminary conclusions tend
toward a positive picture of TeamWave Workplace from a usability
perspective. One could wonder why a research project focused on
sociocultural studies of collaborative learning would include a usability study
of technology in its repertoire. We would argue that if the chosen
collaborative technology hinders a user’s work, then creating all the right
social and cultural conditions would not be enough to avoid frustration and
annoyance on the part of the user. This became even clearer in Guribye’s
study of IDEELS where the OPUSi system was so “unusable” that it clearly
hindered their work practices and created frustration. Guribye’s findings
are also supported by Ekker’s conclusions (see above).
In his investigation of how personality factors effected the attitudes and
behaviour of students participating in VisArt, Morgan (see section 7.12)
found statistically significant differences between the major personality facts
in terms of the use of the learning environment and attitudes towards the
various components of TW. Such findings suggest that future development
of collaborative telelearning environments might find it profitable to directly
link personality type tests directly to real time online changes within the
collaborative learning environment to more closely match students preferred
interaction styles and preferred learning tools. Combining the results of
Morgan’ study with Guribye’s and Ekker’s might enable us to make some
interesting observations about how individual student’s will experience online
learning. This, however, is presently speculation, and an integrated analysis
would be necessary before any conclusions could be drawn.
In his research on how to use electronic data logs to support collaborative
telelearning research, Meistad (see section 7.15) has developed two tools
52
“Traditional” is used to indicate that there is a tradition of using usability studies
to determine whether or not a software system enables a user to carry out their
tasks. Traditionally these were laboratory-based studies.
248
Chapter 9 Summary and Conclusions
that present unwieldy log data in a format that is useful for determine
collaboration opportunities and classifying tool use. Using the two tools
in conjunction with other data collection methods will enable researchers
to develop a richer picture of collaboration patterns. Meistad’s tools have
been used by some of the other DoCTA researchers to use the data logs
to complement their own data. For example, Baggetun (see section 7.16)
found that using these tools made it possible to see each team member’s
movements inside TW (e.g., from room to room), to see how long each
team member was logged on, and to see who was logged on at the same
time in a room. This gave an indication of the synchronous/asynchronous
nature of the collaboration within a team.
Finally, we leave behind an archive of artefacts for others to use. As
summarised in table 2, these include: a review of collaborative
technologies; a set of qualitative and quantitative instruments (such as
questionnaires and interview guides); reflections on the use of Activity
Theory; examples of training assignments; examples of a design activity
and how it has been integrated into our courses; documented experience
in the use of TeamWave workplace in an actual collaborative telelearning
activity including how we designed the rooms to support the activity and
examples of the resultant designed learning rooms; a set of TeamWave
Workplace help pages on the web; reflections on administration of
TeamWave workplace; and reflections on the deployment of the VisArt
scenario. We have published and presented VisArt in several forums and
will continue to do so as the preliminary analyses are expanded and
more concrete conclusions can be made.
The DoCTA web site is currently being expanded to include a
demonstration of the VisArt scenario and will periodically be updated as
our activities continue.
9.2
Conclusions
The report began with a presentation of the objectives of DoCTA and
the research questions to be answered. The main, long term, research
objectives are to:
• take a sociocultural perspective on learning activity focussing on the
interpersonal social interaction in a collaborative learning setting
(collaboration = coordination + communication)
249
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
• contribute to collaborative telelearning knowledge about the
pedagogical design of learning scenarios, the technological design
of the learning environment to support these learning scenarios, and
the organisational design for management of such learning
environments, including a reflection on teacher and learner roles
• study and evaluate the social and cultural aspects of distributed
collaborative telelearning environments
In DoCTA, we have maintained a focus on learning activity and have
attempted to integrate this into both the designs and evaluations we have
undertaken. We have focused on both coordination and communication
issues when designing our training and in designing our evaluation studies.
The majority of the evaluation studies we have carried out have roots in
a sociocultural perspective.
The design and deployment of IDEELS and VisArt has not only contributed
to our understanding of the pedagogical, technological and organisational
aspects of collaborative telelearning environments, but has had an impact
on those who have participated as students. These teachers in training
have been given first hand experience in using a collaborative technology and in coordinating and communicating with a team to collaboratively
design a learning artefact. A spin-off result has been that several of the
students have expressed an interest in using the designed learning activity
and the technology (i.e., TeamWave Workplace) in their own teaching.
The numerous evaluation studies that we have carried out not only
contribute to our understanding of the social and cultural aspects of
collaborative telelearning environments, but equally important, they have
also addressed methodological issues related to studying online
environments. As pointed out in section 8.2, the challenges around studying
and understanding the social processes occurring within, and around (Jones, 1999), such environments are a contemporary research challenge.
Two research questions have stood central as we strive to make sense of
these collaborative telelearning environments:
How do students, teachers and facilitators organise their work?
What collaboration patterns emerge?
250
Chapter 9 Summary and Conclusions
Although we are in the early stages of understanding how the participants
in our scenarios organised their work, we feel we are beginning to make
progress. One complete study, Guribye (1999), has documented how
one group of students organised their work in the IDEELS scenario.
Results of this study have informed the studies of Andreassen (in preparation) and Wake (in preparation) who take a similar approach to studying
students work organisation and instructor/facilitator work organisation
in VisArt, respectively. Collaboration patterns are beginning to emerge
from these and the other evaluation studies. Although we are unfortunately
not as far along in our integrative analysis as we would have liked to be,
we feel that the progress we have made thus far is promising. Future
directions for work on collaboration patterns have been enumerated in
section 8.1.
A final aspect of the DoCTA research project that warrants mention is
the fostering of enthusiasm for ICT and learning research both at the
Department of Information Science and in the research milieu in Bergen
in general. We have had little problem in attracting graduate students to
the project and are proud to have initiated, in at least three of the Masters students, an interest in pursuing doctorate studies in this area. The
opportunity afforded us to work on such an encompassing project has
been well worth the efforts.
Finally, we conclude by saying that the research ITU funding has enabled
us to engage in has been both exciting — and challenging!
251
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
252
References
References
Adrianson, L. Hjelmquist, E. (1991) Group processes in face to face
and computer mediated communication. Behaviour and
Information Technology, 10(4), 281-296.
Alexander, C.A., Ishikawa, M. and Silverstein, S. (1977). A Pattern
Language: Towns, Buildings, Construction. Oxford Univerity Press.
New York,
Andreassen, E. F. (in preparation). Evaluating how students organise
their work in a collaborative telelearning scenario: An Activity
Theoretical Perspective. Masters dissertation, Department of
Information Science, University of Bergen, Norway
Andriessen & Sandberg, (1999). Where is education heading and how
about AI? International Journal of Artificial Intelligence in
Education, 10, 130-150.
Badre, A. N., Guzdial, M., Hudson, S. E. and Santos, P. J. (1995). A
user interface evaluation environment using synchronized
video, visualizations and event trace data. Software Quality
Journal 4, pp 101-113.
Baggetun, R. (in preparation). Coordination Work in Collaborative
Telelearning. Masters dissertation, Department of Information
Science, University of Bergen, Norway
Barbuto, J.E. Jr. (1997). A critique of the Myers Briggs Type Indicator
and its operationalization of Carl Jung’s psychological types.
Psychological Reports, 80(2), 611-625.
Bellotti, V. and Bly, S., (1996). Walking away from the desktop
computer: Distributed Collaboration and Mobility in a Product
Design Team. Proceedings of the ACM Conference on Computer
Supported Cooperative Work (CSCW’96), 209-218.
253
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Bentley, R., Appelt, W., Busbach, U., Hinrichs, E., Kerr, D., Sikkel,
K., Trevor, J. & Woetzel, G. (1997). Basic Support for Cooperative Work on the World Wide Web, International Journal of
Human Computer Studies: Special issue.
Bentley, R., Horstmann, T., & Trevor, J. (1997). The World Wide Web
as Enabling Technology for CSCW: The Case of BSCW, Computer
Supported Cooperative Work (CSCW), 6 (2/3), 111-134.
Amsterdam: Kluwer Academic Publishers.
Bourdeau, J. & Bates, A (1997). Instructional Design for Distance
Learning. In R. Tennyson. S. Dijsktra, N. Seel, and F. Schott (Eds.)
Instructional Design: International Perspectives, 369-397.
Hillsdale, NJ: LEA Publishers.
Bourdeau, J. & Wasson, B. (1997). Orchestrating collaboration in
collaborative telelearning. In Proceedings of the 8th World
Conference on Artificial Intelligence in Education, pages 565567. Amsterdam: IOS Press.
Bredo, E. (1994). Reconstructing Educational Psychology: Situated
Cognition and Deweyian Pragmatism. In Educational Psychologist,
29(1), 23-35.
Brown, Virginia L.; DeCoster, David A. (1991). The Myers Briggs
Type Indicator as a developmental measure: Implications for
student learners in higher education. Journal of College Student
Development, 32(4): 378 379
Carroll, J.M., Kellogg, W.A., & Rosson, M. B. (1991). The taskartefact cycle. In J. M. Carroll (Ed.) Designing Interaction:
Psychology at the Human-Computer Interface, 74-102.
Cambridge, UK: Cambridge University Press.
Carstensen, P.H. & Sørensen, C. (1996). From the social to the
systematic? An analysis of mechanisms supporting coordination
work in design. CSCW Journal, 5(4), 387-413.
Cattell, R. (1936). A guide to mental testing. London.
Chabert, A. (1997). Habanero and the Vrteach Educators, http://
havefun.ncsa.uiuc.edu/habanero/Whitepapers/vrteacher.html
254
References
Chabert, A., Grossman, Ed., Jackson, L. & Pietrovicz, S. (1996).
Synchronous collaborative framework and environment, http://
www.ncsa.uiuc.edu/SDG/Software/Habanero/Whitepapers/
ecscw-habanero.html
Chabert, A., Grossman, E., Jackson, L., Pietrovicz, S. & Seguin, C.
(1998). Java Object-Sharing in Habanero. Communications of the
ACM, 6, 69-76.
Chan, T-W. & Baskin, A.B. (1990). Learning Companion Systems. In
C. Frasson, G. Gauthier, (eds.), Intelligent Tutoring Systems: at
the crossroad of Artificial Intelligence and Education, 6-33.
Ablex, Norwood: NJ..
Cole, M. & Engeström, Y. (1993). A Cultural-Historical Approach to
Distributed Cognition. In G. Salomon, (Ed.) Distributed Cognitions,
Psychological and Educational Considerations. 1-46. Cambridge,
Mass.: Cambridge University Press.
Cole, M. & Scribner, S. (1978). Introduction. In Cole, M., JohnSteiner, V., Scribner, S., & Souberman, E. (Eds.). L. S. Vygotsky,
Mind in Society: The Development of Higher Psychological
Processes,1-16. Cambridge, MA: Harvard University Press.
Daniels, H. (1996). An Introduction to Vygotsky. London: Routledge.
Davydov, V. (1988). Learning activity: The main problems needing
further research. Activity Theory, 1(1-2), 29-36.
Decortis, F., Noirfalise, S. & Saudelli, B. (1996). Activity Theory as
framework for cooperative work. COTCOS – T. M. R programme,
WP1.
Dembo, MH, McAuliffe, T.J. (1987) .Effects of Perceived Ability and
Grade Status on Social Interaction and Influence in Cooperative
Groups. Journal of Educational Psychology,79(4), 415-423.
Dillenbourg, P., Self, J.A. (1992). People Power: A Human –
Computer Collaborative Learning- System. Lecture Notes in
Computer Science, 608, 651-660.
Doise, W. (1990). The development of individual competencies
through social interaction. In H.C. Foot, M.J. Morgan & R.H. Shute
(Eds.) Children helping children, 43-64. Chichester: J. Wiley and
Sons.
255
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Dourish, P., and Bellotti, V.(1992), Awareness and Coordination in
Shared Workspaces, Proceedings of the Conference on
Computer-Supported Cooperative Work, 107-114.
Eidsmo, A., Munkvold G. & Ekker, K. (1999). (Nord-Trondelag
College, Norway): Experiences with TeamWave and OPUSi
software as pedagogical tools in CSCL (computer supported
cooperative learning). http://www.unice.fr/RI/Journees-TIC/
(Démonstrations)
Eison, J.A., Pollio, H.R. (1985). A multidimensional approach to the
definition of college students’ learning styles. Journal of College
Student Personnel. 26(5), 434-443.
Engeström, Y (1987). Learning By Expanding: An activitytheoretical approach to developmental research. Helsinki:
Orienta-Konsultit Oy
Engeström, Y. (1991). Activity theory and individual and social
transformation. Activity Theory 7(8), 6-17.
Engeström, Y., Miettinen, R. & Punamäki, R-L. (1999). Perspectives
on Activity Theory. Cambridge, UK: Cambridge University Press.
Fitzpatrick, G., Tolone, W.J., Kaplan, S.M. (1995). Work, locales and
distributed social worlds. In H. Marmolin, Y. Sundblad & . Schmidt
(Eds.) Proceedings of the Third European Conference on
Computer-Supported Cooperative Work (ECSCW’95), 1-16.
Dordrecht: Kluwer Academic Publishers.
Fjeld, M., Schluep, S., Rauterberg, M. (1998). Automatic, action
driven classification of user problem solving strategies by statistical
and analytical methods: a comparative study. In F. E. Ritter and R.
M. Young (eds.) Proceedings of the Second European Conference
on Cognitive Modelling, 98-103. Nottingham: Nottingham
University Press.
Fjuk, A. (1998). Computer Support for Distributed Collaborative
Learning. Exploring a Complex Problem Area. Dr. Scient. Thesis.
University of Oslo, Department of Informatics.
Fjuk, A., Sorensen, E. & Wasson, B. (1999). ICT-mediated learning
and competence development in companies. NIN pre-project final
report. Telenor FOU, Kjeller, Norway.
256
References
Fjuk, A & Øgrim, L. (1997). Towards transcendent thinking in design
of distributed collaborative learning. Accepted for publication in
European Journal of Open and Distance Learning.
Gamma, E.R., Helm, R., Johnson, R. & Vlissides, L. (1995). Design
Patterns: Elements of Re-usable Object-Oriented Software. Reading
MA: Addison-Wesley.
Garden, A.M. (1991). Unresolved issues with the Myers Briggs Type
Indicator. Journal of Psychological Type, 22, 3-14.
Garton, L., Haythorntwaite, C. & Wellman, B. (1999). Studying online social networks. In S. Jones (Ed.) Doing Internet research:
Critical issues and methods for examining the net, 75-106.
London: Sage Publicaions, Inc.
Gibson, J. (1986). The ecological approach to visual perception.
Hillsdale NJ: Lawrence Erlbaum Associates.
GMD (1999). What is BSCW? http://bscw.gmd.de/index.html
Guribye, F. (1999). Evaluating a collaborative telelearning scenario: A
sociocultural perspective. Masters dissertation, Department of
Information Science, University of Bergen, Norway.
Guribye, F. & Wasson B. (1999). Evaluating Collaborative Telelearning
Scenarios: A Sociocultural Perspective. In B. Collis & R. Oliver
(Eds.) Proceedings of Educational Multimedia & Educational
Telecom ’99, 1264-1265. Charlottesville, VA: AACE. (ISBN : 1880094-35-5) (Outstanding Paper Award). http://www.uib.no/
People/stud1068/documents/edmedia.html
Guribye, F. & Wasson, B. (submitted). Using Activity Theory to
Understand Work Organisation in a Collaborative Telelearning
Scenario. Submitted to the International Journal on Learning
Environments Research, Kluwer.
Gutwin, C., Stark, G. and Greenberg, S. (1995). Support for
Workspace Awareness in Educational Groupware. in Proceedings
of the ACM Conference on Computer Supported Collaborative
Learning 147-156. Hillsdale NJ: Lawrence Erlbaum Associates.
Guzdial, M. (1993). Deriving software usage patterns from log files.
Technical Report #GIT-GVU-93-41. Graphics, Visualization, and
Usability Center, College of Computing, Georgia Insitute of Technology.
257
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Habanero (1998). Habanero tools. http://www.ncsa.uiuc.edu/SDG/
Software/Habanero/Tools/
Holt, A. (1985).. Coordination technology and petri nets. In G.
Rozenberg (Ed.) Advances in Petri Nets 1985, 222, 278-296.
Holt, A.W. (1998). DIPLANS – A new Language for the Study and
Implementation of Coordination. ACM Transactions in
Information Systems. 6(2), 109-125.
Holzman, L. H. (1996). Pragmatism and Dialectical Materialism in
Language Development. In H. Daniels (Ed.). An Introduction to
Vygotsky. London: Routledge.
Hutchins, E. (1991). The social organisation of distributed cognition. In
L. Resnick, J. Levine, & S. Teasley (Eds.). Perspectives on Socially
Shared Cognition, 238-287. Washington, DC: American
Psychological Association Press.
Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA: MIT
Press.
Jackson, Stacy L.; Parker, Christopher P.; Dipboye, Robert L. (1996).
A comparison of competing models underlying responses to the
Myers Briggs Type Indicator. Journal of Career Assessment. 4(1),
99-115.
Johansen, R., Sibbet, D., Benson, S., Martin, A., Mittman, R. & Saffo,
P. (1991). Leading Business Teams. New York: Addison-Wesley.
JohnSteiner, V, Mahn, H. (1996). Sociocultural approaches to learning
and development: A Vygotskian framework Educational
Psychologist, 31( 3-4), 191-206.
Jonassen, D. & Rohrer-Murphy, L. (1999). Activity Theory as a
Framework for Designing Constructivist Learning Environments.
Educational Technology (47).
Jones, S.G. (Ed.) (1999). Doing Internet Research: Critical Issues and
Methods for Examining the Net. London: SAGE Publications Ltd.
Jung, C. (1953 –1971) Collected works. London.
Junge, A. (1999). Hva er The Demeter Project ? Masters
dissertation, Department of Information Science, University of
Bergen, Norway
258
References
Kaplan, S.M., Tolone, W.J., Bogia, D.P. & Bignoli, C. (1992).
Flexible, active support for collaborative work with conversation
builder. In J. Turner & R. Kraut (Eds.) Proceedings of the
Conference on Computer-Supported Cooperative Work
(CSCW’92), 378-385. New York: ACM Press.
Kaptelinin, V. & Nardi B. A. (1997). The Activity checklist: A tool for
representing the “space” of context. Working paper. Department
of Informatics, Umeå University.
Kendall, L. (1999) Recontextualizing “Cyberspace”: Methodological
considerations for on-line research, 57-74. In S.G. Jones (Ed.)
Doing Internet Research: Critical Issues and Methods for
Examining the Net. London: SAGE Publications Ltd.
Kerr, N.L. (1983). Motivation losses in small groups: A social dilemma
analysis. Journal of Personality and Social Psychology, 45, 819828.
Kerr, N.L. & Bruun, S.E. (1983). Dispensability of member effort and
group motivation losses: Free rider effects. Journal of Personality
and Social Psychology, 44, 78-94.
Koschmann, T. (1996). Paradigm shifts and instructional technology:
An introduction. In T. Koschmann (Ed.) CSCL: Theory and Practice
of an Emerging Paradigm, 1-23. Mahwah, NJ: Lawrence Erlbaum
Associates.
Kuutti, K. (1996). Activity theory as a Potential framework for humancomputer interaction research. In B.A. Nardi (Ed.) Context and
Consciousness: Activity Theory and Human-computer
Interaction, 17-44. Cambridge, MA: MIT Press.
Laurillard, D. (1987). Computers and the Emancipation of Students –
Giving Control to the Learner. Instructional Science, 16(1), 3-18.
Lave, J. (1988) Cognition in Practice. Cambridge University Press.
Lave, J. & Wenger, E. (1991). Situated Learning: Legitimate
Peripheral Participation. NY: Cambridge University Press.
Lave, J. & Wenger, E. (1996). Practice, Person, Social World. In H.
Daniels (Ed.). An Introduction to Vygotsky. London: Routledge.
259
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
LDC (1998a). White Paper LearningSpace, Solutions for Anytime
Learning”, Lotus Development Corporation, 55 Cambridge
Parkway, Cambridge, MA 02142
LDC (1998b). Fact Sheet. Lotus LearningSpace Release 2.5
Distributed Learning Solution. Lotus Development Corporation,
55 Cambridge Parkway, Cambridge, MA 02142
Leont’ev, A. N. (1978). Activity, Consciousness, Personality.
Englewood Cliffs, NJ: Prentice Hall.
Lorr, M. (1991). An empirical evaluation of the MBTI typology.
Personality and Individual Differences 12(11), 1141-1145.
Malone, T. & Crowston, K. (1994). The Interdisciplinary study of
coordination. ACM Computing Surveys, 26(1), 87-119.
Mantovani, G. (1996). New Communication Environments: From
Everyday to Virtual. London: Taylor & Francis
Marton, F. & Booth, S. (1997). Learning and awareness. Hillsdale,
N.J.: Lawrence Erlbaum Associates.
Mead, G. H. (1968). Mind, Self, and Society. Chicago: Chicago
University Press.
Meistad, Ø. (in preparation). Collaborative telelearning: Using log-files
to identify collaboration patterns. Masters dissertation,
Department of Information Science, University of Bergen.
Meistad, Ø. & Wasson, B. (submitted). Supporting collaborative
telelearning research using server logs. Submitted to of
Educational Multimedia & Educational Telecom 2000.
Minick, N. J. (1996). The Development of Vygotsky’s Thought: An
Introduction to Thinking and Speech. In H. Daniels (Ed.). An
Introduction to Vygotsky. London: Routledge.
Morgan, K., Macleod, H. (1992). Results from exploratory
investigations into the possible role of personality factors in
computer interface preference. Paper presented to the UK Joint
Research Council’s Second Interdisciplinary Workshop on
Mental Models, Robinson College, University of Cambridge, March
1992
260
References
Morgan, K. & Morgan, M. (1999). Religion and the Internet: the
cultural impact of information technology on culture and religious
beliefs. Proceedings of the VI European Congress of Psychology,
Rome, Italy July 4-9 1999.
Morgan, K. & Morgan, M. (in preparation). The Role of Classical
Jungian Personality Factors in Computer Supported Collaborative
Learning Environments.
Murray, J.B. (1990). Review of research on the Myers Briggs Type
Indicator. Perceptual and Motor Skills, 70(3, Pt 2), 1187-1202.
Mørch, A. (1995). Application Units: Basic Building Blocks of
Tailorable Applications. Proceedings 5th International East-West
conf. on HCI, Moscow, July 1995. Lecture Notes in Computer
Science 1015, 68-87. Amsterdam: Springer Verlag.
Nardi, B. A. (1996). Studying Context: A comparison of activity theory,
situated action models and distributed cognition. In B.A. Nardi, (Ed.)
Context and Consciousness: Activity Theory and Humancomputer Interaction, 69-102. Cambridge, MA: MIT Press.
Newhagen, J. E., and Rafaeli, S. (1997). Why communication
researchers should study the Internet: A dialogue [On-line]. Journal
of Computer-Mediated Communication, 1(4). Available: http://
www.ascusc.org/jcmc/vol1/issue4/rafaeli.html.
Norman, D.A. (1990). Human Error and the Design of Computer
Systems – Commentary. Communications of the ACM, 33(1), 4.
Okada, H., Asahi, T. (1999). GUITESTER: A log-based usability testing
tool for Graphical User Interfaces. IEICE Transactions on
information and systems. Volume E82.
Olson, J., Card, S., Landauer, T., Olsen, G. Malone, T. & Legget, J.
(1993). Computer-supported cooperative work: Research issues for
the 90s. Behavior and Information Technology, 12 (2), 115-129.
Palfreyman K. A. and Rodden T. (1996). A Protocol for User
Awareness on the World Wide Web, In Proceedings of CSCW’96 Cooperating Communities,130 – 139. ACM Press.
Paquette, G, Ricciardi-Rigault, C., Bourdeau, J., Paquin, C. &
Liegeois, S. (1995). Modelling a Virtual Campus Environment for
Interactive Distance Learning. Proceeding of EdMedia’95.
261
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Pea, R. (1993). Practices of distributed intelligence and designs for
education. In G. Salomon (Ed) Distributed Cognitions: Psychological
and Educational Considerations, 47-87. Cambridge, Mass:
Cambridge University Press.
Pea, R., Edelson, D., & Gomez, L. (1994). Distributed collaborative
science learning using scientific visualization and wideband
telecommunications. Paper presented at the symposium entitled
Multimedia information systems for science and engineering
education: Harnessing technologies at the 160th Meeting of the
American Association for the Advancement of Science (AAAS), San
Francisco.
Pedersen, T. (in preparation). Technological artefacts and collaboration
patterns: evaluating the use of groupware in a collaborative
telelearning scenario. Masters dissertation, Department of
Information Science, University of Bergen, Norway.
Resnick, L., Levine, J. & Teasley, S. (1991, Eds.). Perspectives on
socially shared cognition. Washington, DC: APA Press.
Richek, H.G. (1969). Jung’s typology and psychological adjustment in
prospective teachers: A preliminary investigation. Alberta Journal
of Educational Research. 15(4), 235-243.
Rogoff, B. (1990). Apprenticeship in Thinking: Cognitive
Development in Social Contexts. New York: Oxford University
Press.
Rysjedal, K. (in preparation). Usability Evaluation of a Groupware
Tool. Masters Dissertation. Department of Information Science,
University of Bergen, Norway
Jonassen, D.H. & RohrerMurphy, L. (1999). Activity theory as a
framework for designing constructivist learning environments.
ETR&D- Educational Technology Research and Development,
47(1), 61-79.
Roseman, M. & Greenberg, S. (1996). TeamRooms: Networked
Places for Collaboration. Proceedings of ACM CSCW’96
Conference on Computer Supported Cooperative Work, ACM
Press.
262
References
Salomon, G. (1992). What does the design of effective CSCL require
and how do we study its effects ? SIGCUE Outlook, Special Issue
on CSCL, 21(3), 62-68.
Salomon, G. (Ed.) (1993). Distributed Cognitions, Psychological and
Educational Considerations. Cambridge: Cambridge University
Press.
Salomon, G, Globerson, T. (1987). The Computer as a Zone of
Proximal Development – Internalizing Reading Related
Metacognitions from a Reading Partner. Journal of Educational
Psychology. 81( 4), 620-627.
Sardamalia, M, Bereiter, C, McLean, RS, Swallow, J, Woodruff, E,
(1987). Computer Supported Intentional Learning Environments.
Journal of Educational Computing Research, 5(1), 51-68.
Schmidt, K (1997). Of maps and scripts. The status of formal
constructs in cooperative work. Proceedings of the ACM
Conference on Supporting Group Work (Group’97), 138-147.
New York: ACM Press.
Schmidt, K. (1994). Cooperative Work and its Articulation Requirements for Computer Support. Travail Humain, 57( 4),
345-366.
Schmidt, K. & Simone, C. (1996). Coordination mechanisms: Towards
a conceptual foundation of CSCW systems design. Computer
Supported Cooperative Work: The Journal of Collaborative
Computing, 5, 155-200.
Scott, T. Cole, M. Engel, M. (1992). Computers and Education - A
Cultural Constructivist Perspective. Review on Research in
Education, 18, 1992, 191-251.
Singleton, W. T. (1989). The mind at work: Psychological ergonomics.
Cambridge, England UK: Cambridge University Press.
Stetsenko, A. P. (1990). The Role of the Principle of Objectrelatedness in the Theory of Activity: Criticism from “Without” and
from “Within”. In V.V. Davydov, & D.A. Leont’ev, (Eds.). The
Activity Approach to Psychology: Problems and Perspectives.
Moscow: USSR Academy of Pedagogical Sciences.
263
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Strauss, A. (1985). Work and the division of labor. The Sociological
Quarterly, 26(1), 1-19.
Suchman, L. (1987). Plans and situated actions. Cambridge
University Press
Suthers, D. & Weiner, A. (1995). Groupware for Developing Critical
Discussion Skills. Proceeding of Computer Supported
Collaborative Learning 1995, October 17-20, 1995, Indiana
University, Bloomington, IN.
Säljö, R. (1996). Mental and physical artifacts in cognitive practices. In
P. Reimann & H. Spada (Eds). Learning in humans and
machines. Towards an interdisciplinary learning science, 83-96.
London: Pergamon.
TeamWave Workplace (1999). TeamWave Workplace. http://
www.teamwave.com/advantages/overview.html
Underhaug, H. (in preparation). Facilitating Help and Training in a
Collaborative Telelearning Scenario. Masters dissertation,
Department of Information Science, University of Bergen, Norway.
Van der Veer, G.C., Tauber, M.J., Waern, Y., Van Muylwijk, B.
(1985). On the interaction between system and user characteristics.
Behaviour and Information 4(4), 289-308.
Van Hoe, R., Poupeye, K., Vandierendonck, A., de Soete, G. (1990).
Some effects of menu characteristics and user personality on
performance with menu driven interfaces. Behaviour and
Information Technology 9(1), 17-29.
Van Muylwijk, B., Van der Veer, G., Waern, Y. (1983). On the
implications of user variability in open systems: An overview of the
little we know and of the lot we have to find out. Behaviour and
Information Technology 2(4), 313-326.
Vygotsky, L. S. (1978)., Mind in Society: The Development of
Higher Psychological Processes M.Cole, V. John-Steiner, S.
Scribner, & E. Souberman (Eds.). Cambridge, MA: Harvard
University Press.
264
References
Wake, J. (in preparation). Evaluating the Organisation of a
Collaborative Telelearning Scenario; An Activity Theoretical
Perspective. Masters dissertation, Department of Information
Science, University of Bergen, Norway.
Wasson, B. (1997). Advanced educational technologies: The learning
environment. Computers in Human Behaviour, 13(4), 571-594.
Wasson, B. (1998). Identifying Coordination Agents for Collaborative
Telelearning. International Journal of Artificial Intelligence in
Education, 9, 275-299.
Wasson, B. (1999). Design and evaluation of a collaborative
telelearning activity. In C. Hoadley (Ed) Proceedings of CSCL99
Designing new media for a new millennium: Collaborative
Technology for Learning,Education, and Training, 659-666.
ACM Press.
Wasson, B. & Mørch, A. (1999). DoCTA: Design and Use of
Collaborative Telelearning Artefacts. In B. Collis & R. Oliver (Eds.)
Proceedings of Educational Multimedia & Educational Telecom
’99, 534-539. Charlottesville, VA: AACE.
Wasson, B. & Mørch, A. (to appear). Identifying collaboration patterns
in collaborative teleleraning scenario. IFETS Special Issue on Online Collaborative Learning Environments.
Weil, M.M., Rosen, L.D., Wugalter, S.E. (1990). The etiology of
computerphobia. Computers in Human Behavior 6(4), 361-379.
Wertsch, J. V. (Ed.) (1981). The Concept of Activity in Soviet
Psychology. Armonk, NY: M.E. Sharpe.
Wertsch, J. V. (1998). Mind as Action. Cambridge: Cambridge
University Press.
Wertsch, J. V., del Río, P. & Alvarez, A. (1995). Sociocultural studies:
history, action and mediation,. In J.V. Wertsch, P. del Río & A.
Alvarez. (Eds.) Sociocultural Studies of Mind, 1-34. Cambridge:
Cambridge University Press.
Zinchenko, V. P. (1995). Cultural-historical psychology and the
psychological theory of activity: retrospect and prospect. In J.V.
Wertsch, P. del Río, & A. Alvarez. Sociocultural Studies of Mind,
37-55. Cambridge University Press.
265
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Åberg-Bengtsson, L. (1998). Entering a Graphicate Society: Young
Children Learning Graphs and Charts. Doctoral thesis.
Gothenburg: University of Gothenburg.
266
Appendicies
Appendicies
Appendix A: DoCTA Publications
Appendix B: DoCTA Presentations
Appendix C: Assignment for test-scenario
Appendix D: Interview Guide for Evaluation of Groupware Systems
Appendix E: The IDEELS Scenario
Appendix F: Assignment for designing an evaluation of IDEELS
Appendix G: Interview Guide for IDEELS
Appendix H: Pre-simulation survey for IDEELS
Appendix I: Post-simulation survey for IDEELS
Appendix J: Planning Tables from the Instructor’s View (Phases II-IV)
Appendix K: Planning Tables from the Researcher’s View (Phases I-III)
Appendix L: Planning Tables from the Student’s View (Phases I-III)
Appendix M: VisArt Consent Form
Appendix N: Semester Assignment
Appendix O: Directions for TeamWave Workplace Installation
Appendix P: Designed Learning Rooms from VisArt
Appendix Q: VisArt Profile Questionnaire
Appendix R: VisArt Self-Evaluation Questionnaire
Appendix S: VisArt Pre-questionnaire
Appendix T: VisArt Post-questionnaire
Appendix U: The Narg Island Game
Appendix V: Norweign Version of Training Assignments 1,3 & 4
Appendix W: Help Page Screen Shots
Appendix X: Administration Interface in TeamWave Workplace
Appendix Y: Usability Study Interview Guide
Appendix Z: Interview Guide for How Students Organise their Work
Appendix AA: Interview Guide for How Instructors and Facilitators
Organise their Work
267
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix A:
DoCTA Publications
Guribye, F. (1999). Evaluating a collaborative telelearning scenario: A
sociocultural perspective. Masters dissertation, Department of
Information Science, University of Bergen, Norway. Available as
EIST Research Report 4.(http://www.ifi.uib.no/docta/thesis/)
Guribye, F. & Wasson, B. (1999). Evaluating collaborative telelearning
scenarios: A sociocultural perspective. In B. Collis & R. Oliver (Eds.)
Proceedings of Educational Multimedia & Educational Telecom
’99, 1264-1265. Charlottesville, VA: AACE. (http://
www.ifi.uib.no/staff/frode/publications/edmedia99.html)
(Outstanding Paper Award)
Meistad, Ø, (1999). Skolens bruk av kulturgjenstander med innebygd
intelligens. Essay, ITU- Interactive Conference (ITU- Konferansen,
1999), Oslo, Norway, October 8th. (http://www.ifi.uib.no/docta/
resources/ITUessay99.html)
Meistad, Ø. & Wasson, B. (to appear). Supporting collaborative
telelearning research using server logs. To appear at the
International Conference on Educational Multimedia &
Educational Telecom 2000, Montreal, June.
Mørch, A. (1999). Project DoCTA: Knowledge-Building in
Collaborative Telelearning, Position paper at the Workshop on
Collaborating on the Design and Assessment of KnowledgeBuilding Environments in the 2000’s, CSCL’99, Stanford CA,
12-15 December. (http://www.cs.colorado.edu/%7Egerry/
publications/conferences/1999/cscl99/kbe_workshop/
morch.html)
Mørch, A. & Wasson, B. (1999). Dynamics of groupware use in a
collaborative telelearning scenario. Position paper at Workshop on
«Evolving use of groupware» at ECSCW’99, Copenhagen DK, 12
September, 1999.
268
Appendicies
Wasson, B. (1999a). Design and evaluation of a collaborative
telelearning activity. In A. Gulbrandsen (Ed.) UPED Report Series
nr. X/99, Programme for Research on Learning and Instruction,
University of Bergen.
Wasson, B. (1999b). Design and evaluation of a collaborative
telelearning activity. In C. Hoadley (Ed) Proceedings of CSCL’99
Designing new media for a new millennium: Collaborative
Technology for Learning,Education, and Training, 659-666.
ACM Press.
Wasson, B. & Mørch, A. (1999). DoCTA: Design and Use of
Collaborative Telelearning Artefacts. In B. Collis & R. Oliver (Eds.)
Proceedings of Educational Multimedia & Educational Telecom
’99, 534-539. Charlottesville, VA: AACE. (http://www.ifi.uib.no/
staff/barbara/papers/edmedia99.html)
Wasson, B. & Mørch, A. (to appear). Identifying collaboration patterns
in collaborative teleleraning scenario. IFETS Special Issue on Online Collaborative Learning Environments.
Submitted Paper
Guribye, F. & Wasson, B. (submitted). Using Activity Theory to
Understand Work Organisation in a Collaborative Telelearning
Scenario. Submitted to the International Journal on Learning
Environments Research, Kluwer.
269
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix B:
DoCTA Presentations
Guribye, F. (1999, December). Design and evaluation of a
collaborative telelearning activity aimed at teacher training.
Presented at CSCL’99. San Francisco.
Mørch, A. (1999, December). Project DoCTA: Knowledge-Building in
Collaborative Telelearning, Presented at the Workshop on
Collaborating on the Design and Assessment of KnowledgeBuilding Environments in the 2000’s, CSCL’99 , Stanford CA.
Munkvold, G., Eidsmo, A. & Ekker, K. (November, 1999).
Experiences with TeamWave Workplace and OPUSi software as
pedagogical tools for CSCL. Journées d’études internationales,
Nice et Sophia Antipolis, France.
Baggetun, R. & Guribye, F. (1999, November) The Use of TeamWave
Workplace to Support Collaborative Learning in VisArt. Presented
at Workshop on ICT- Mediated Learning, Dragefjellet, University
of Bergen, Norway.
Morch, Anders (1999, October). DoCTA/VisArt: Use of Groupware in
Design of Visual Artefacts. ITU- Interactive Conference (ITUKonferansen), Oslo, Norway, October 8th.
Guribye, Frode (1999, October). Virtual Ethnography. ITU-Interactive
Conference (ITU- Konferansen), Oslo, Norway, October 8th.
Meistad, Øyvind (1999, October). Log-files as a Supplement to the
Evaluation of Collaborative Telelearning Scenarios. ITU- Interactive
Conference (ITU- Konferansen), Oslo, Norway, October 8th.
Munkvold, G., Eidsmo, A. & Ekker, K. (September, 1999). Virtual
Collaboration: experiences from an international and a national
project: Project Ideels and Project Docta. Presentation at the NordTrøndelag College, Research and Development Day.
270
Appendicies
Mørch, Anders (1999, September). Activity Theory: Basic Concepts
and their Evolution. Invited presentation at Workshop on Evolving
use of Groupware at European Conference on Computer
Supported Cooperative Work (ECSCW99), Copenhagen,
Denmark, September 12th.
Spector, Mike & Guribye, Frode (1999, August). Theoretical
Foundations for the design of collaborative distance learning /
VisArt. European Conference for Research on Learning and
Instruction (EARLI’99). Göteborg, Sweden, August 26th.
Guribye, Frode (1999, June). Evaluating collaborative telelearning
scenarios: A sociocultural perspective. International Conference
on Educational Multimedia and Telecommunications, Seattle,
WA, June 23rd.
Guribye, Frode (1999, June). Project DoCTA: Design and use of
Collaborative Telelearning Artefacts. International Conference on
Educational Multimedia and Telecommunications, Seattle, WA,
June 21st.
Wasson, Barbara (1999, June). VisArt: A collaborative telelearning
scenario. Invited speaker at the LINGO Project Workshop,
University of Bergen, June 4th.
Wasson, Barbara (1999, April). Project DoCTA. ITU Group Leader
Workshop, University of Oslo, April 26th.
Wasson, Barbara (1999, April). Reflections on Activity Theory. ITU
Group Leader Workshop, University of Oslo, April 26th.
Wasson, Barbara (1998, October). Project DoCTA. ITU Konferanse:
Fra tradisjon til innovasjon - IKT og utdanning, University of
Oslo, October 13th.
Guribye, Frode (1998, October). Project DoCTA. Human Centred
Technology Workshop, University of Sussex, Brighton, UK,
October 3rd.
271
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix C:
Assignment for test-scenario (English)
“Ytrebygda Tannlegesenter” is a private dental clinic with 18 employees.
There are 4 secretaries, 4 dental-assistants, 5 dentists and 5 assistants.
Every dentist has a personal assistant, but sometimes it will be necessary
for the assistant to have a day off. It will then be possible to switch with
some other assistant, if time allows.
The clients have a regular dentist, whom the consult. But sometimes it
can be necessary to seek out different competence. One wishes to provide all kind of dental treatment on the clinic. As for now, they have about
2500 regular patients, and a number of emergency-patients who only
come this one time. They are also provided with a number of new patients,
often children of their regular customers. It is not desirable to get too
many new customers. A number of the patients are long-term customers
who come once a month to get braces or orthodontic treatment.
There has been some trouble with the manual archives, especially when
a patient has received treatment from different dentists. The files have a
tendency to stay in the different dentists’ own little archives, instead of
being put in the main archive in the secretaries’ office. This leads to
much redundancy of information, and that not all-important information
reaches the archive. This can lead to serious consequences on what is to
be done to the patients. One wishes to have a better archive-system,
which is more accessible and more informative.
Also, there is no complete overview of the work-schedule of the different
dentists, this because the dentists give their patients new appointments
without informing the secretaries. This can lead to double booking of
appointments and great frustration with the patients and secretaries.
The same problems are to be found by the dental-assistants. In turn,
these problems lead to disputes between dentists and dental-assistants
because nobody knows exactly when the others have given the patients a
new appointment. One wishes to have better overview, as to the patients
to be more dispersed between the different dentists. This will lead to
better time managing.
272
Appendicies
The board of the clinic has decided to grant some funding to get an
improved system. Exactly how much is not yet decided, but they want to
look at several different proposals before they make up their mind.
Give a proposal of how these problems can be solved. Make a use-case
diagram and a class diagram. Use UML-notation if possible. It is not
necessary to include methods and variables.
Oppgave til test-scenario (Norsk)
Bedriften ”Ytrebygda Tannlegesenter” er en privatdrevet tannklinikk med
18 ansatte. Man har 4 sekretærer, 4 tannpleiere, 5 tannleger og 5 assistenter. Hver tannlege har sin egen assistent, men enkelte ganger kan
det være nødvendig for assistentene å ha fri. Man kan da bytte med
noen av de andre assistentene dersom det er mulig tidsmessig.
Klientene har sin egen faste tannlege, men noen ganger kan det være
aktuelt med annen kompetanse enn det den faste tannlegen har. Man
ønsker på klinikken å kunne gi alle former for tannbehandling. For tiden
har man en krets på omtrent 2500 faste klienter, og en del hastesaker
som kun er engangstilfeller. Man får også en del nye klienter etterhvert,
da gjerne barn av dem som allerede er faste kunder. Man ønsker ikke å
ta inn så veldig mange flere faste kunder. En rekke av kundene er langtidskunder som kommer hver måned på grunn av streng/tannstillings-behandling.
Man har hatt en del problemer med de manuelle arkivene, da spesielt
når en pasient har vært hos flere ulike tannleger på senteret. Mappene
har en tendens til å havne i den enkelte tannleges eget lille arkiv, i stedet
for i hovedarkivet som befinner seg på sekretærenes kontor. Dette fører
også til mye dobbeltlagring av informasjon, og at ikke all informasjon
havner i mappene. Slike ting kan gi alvorlige konsekvenser på det som
gjøres med pasientene. Man ønsker et bedre system på arkivene, slik at
de blir mer tilgjengelig, og mer informative.
Man har heller ikke noe fullgod oversikt over alle tannlegenes arbeidsplan, da de gjerne gir pasientene ny time, uten å informere sekretærene
om dette. Det kan gi dobbelt-bestilte timer, til pasientenes og sekretærenes store frustrasjon. De samme problemene går igjen hos tannpleierne.
273
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Det blir også en del gnissninger mellom tannpleiere og tannleger, ettersom ingen vet helt når de andre har gitt pasientene timer. Man ønsker å
kunne ha bedre oversikt, slik at pasientene blir jevnere spredd ut på de
ulike tannlegene. Dette vil også gi bedre disponering av tid.
Tannlegekontoret har besluttet å sette av en del midler i forbindelse med
å få et forbedret system. Akkurat hvor mye er ikke klart, men de ønsker
å se på en del ulike løsninger før de bestemmer seg.
274
Appendicies
Appendix D:
Interview Guide for Evaluation of Groupware Systems
(English)
1. What expectations did you have to a groupware-system before
conducting this test?
2. Did this program fulfil your expectations? Why/ Why not?
3. What surprised you more, solving a task like this, using the program?
4. Did you feel you could solve the task in the way you are used to, or
was it more/ less demanding?
5.
6.
7.
8.
9.
Did you feel you had more limitations or more possibilities?
How did you find the user interface of the program?
Did you find the program difficult or easy to understand?
Have you any suggestions for changes/improvements of the program?
Did you miss something in this program?
Intervju-guiden (Norsk)
1. Hvilke forventninger hadde du til et gruppevare-program i forkant av
denne testen?
2. Levde dette programmet opp til dine forventninger? Hvorfor / hvorfor ikke?
3. Hva overrasket deg mest med å løse oppgaver på denne måten (vha
programmet)?
4. Følte du at du kunne løse oppgavene slik du er vant med, eller ble det
mer/mindre krevende?
5.
6.
7.
8.
9.
Følte du at du fikk begrensninger, eller flere muligheter?
Hva synes du om brukergrensesnittet i programmet?
Fant du programmet vanskelig eller enkelt å forstå?
Forslag til endringer/forbedringer?
Var det noe du savnet i programmet?
275
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix E:
The IDEELS Scenario
Scenario: Designing the Tertiary Education System
How IDEELS Simulations Work
In IDEELS simulations, students act as high-level negotiators, consultants,
and journalists in a fictional world, working to resolve real problems of
importance to the European Community – problems which can only be
solved through cooperation at the international level. In so doing,
participants model real interactions between nations and organizations.
The simulations are conducted on two levels: deliberations within a team
and negotiations between or among teams. Within a team, students
familiarize themselves with their assigned profile (country, consultant
group, or journalists) and work together to formulate the positions they
will present in the multilateral negotiations. They then communicate their
policies and conduct a dialogue with the other country teams and consultant groups by sending written messages over a computer-based
communications system, OPUSi. In addition, teams confer with each other
during real-time, online teleconferences. While the focus of the exercise
is multilateral, participants should keep in mind that bilateral discussions
are also a central component of the negotiations. The goal is to produce
and ratify a jointly-written document (in this case, a set of
recommendations) that addresses the problems set forth in the scenario.
As a participant in an IDEELS simulation, you will be a member of a
team representing one of the following: a country, a team of technical
consultants, or a team of journalists.
Work with your teammates to prepare your positions carefully, looking
for a balance between national, regional, or local strategic considerations
and international (Federation) interests.
276
Appendicies
In the simulation, which is set in the early years of the 21st century, you
will engage in the following processes or phases:
Phase 1: Pre-Negotiation Your team will prepare two documents:
1) an Internal Briefing Document and
2) a Policy Statement
Both of these should be ready to submit at the beginning of the
simulation. The Internal Briefing Document is intended to guide your
team’s actions during the negotiations; it outlines your positions on the
issues, your team’s interests, negotiating goals, and strategies for
achieving these goals. This document is confidential and should
not be revealed to other teams. The Policy Statement , in contrast, is for public consumption and sets out your team’s initial
negotiating position – its views concerning the issues to be discussed,
their significance for your country or organization as well as for Eutropia,
and ideas that you would like to see included in the final document.
Phase 2: Opening Positions
By noon CET on the Thursday before the negotiations begin (November 4), all teams should have finished writing their Internal Briefing
Documents and Policy Statements. The Internal Briefing Document
should be put in “Papers” in OPUSi. A copy of this Internal Briefing
Document is to be e-mailed to the simulation moderator:
[email protected]
but not to the other teams. Policy Statements, on the other hand,
should be sent to all other teams (via the OPUSi Message Centre) by
the same time.
Phase 3: Dialogue
Once you have read the other teams’ Policy Statements, you should
immediately begin writing messages (Message Centre) to other teams to
request clarification of any points that are not clear or to suggest areas
of possible agreement. The Eutropian Commissioner will schedule a
teleconference during the first week to open the negotiations.
277
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Phase 4: Negotiation
Throughout the simulation, teams will communicate with each other via
the Message Centre and will participate in bilateral and multilateral
teleconferences. Most of the real work of the negotiations takes place
via the Message Centre’s asynchronous messaging system. The scheduled
teleconferences provide opportunities for negotiators to discuss specific
issues with all participants simultaneously.
Phase 5: Closure
The agreed-upon recommendations are to be submitted to the Eutropian
Commissioner and ratified at a final teleconference.
Phase 6: Debriefing
Both during and following the simulation, you will have opportunities to
reflect on and discuss your participation in the simulation.
Guidelines for Writing the Internal Briefing Document
A well-conceived briefing document is essential for the success of an
IDEELS simulation. Writing a briefing document gives you a chance to
learn more about the topics and issues that are central to the simulation.
It also allows teams to formulate their policy goals and plan negotiating
strategies before the simulation begins. This guide will help you prepare
your internal briefing document.
I. Overall format and organisation of the Internal Briefing
Document
A. The Internal Briefing Document is the product of a group effort. Therefore, it should be submitted as a unified document rather than as separate submissions from individuals or from issue area working groups.
1. The final text should be contained in a single document. This document
should be written and revised in “Papers” with a subject line “Internal
Briefing Document - Top Secret!”
278
Appendicies
2. The format, i.e. margins, etc., of the document should be consistent
throughout.
B. The Internal Briefing Document should be as professional a document
as possible. It should contain the following components:
1. A table of contents.
2. An executive summary or overview, which succinctly presents the
following:
a. The overall national policy goals.
b. The broad policy goals for each issue area.
c. A brief description of the overall national negotiating strategy.
C. Finally, the briefing document should not contain any policy
recommendations that are contradictory.
NOTE: The Internal Briefing Document for teams of journalists should
include the following information: What kind of readers do you want to
appeal to? What kind of publication is likely to be taken seriously by
high-level negotiators and high-powered technical consulting groups? What
is your publication’s editorial stance on education-related issues? When
and how often will you publish your newspaper (e.g. daily, three times a
week, or once a week, with special editions or news bulletins as events
require)?
NOTE for technical consultant groups: To a large extent, the
company’s profile, as well as your objectives and goals, are set down in
your team profile and in the call for proposals. Your goal is to have your
technology proposal endorsed by the Task Force on Education and Training (TaskFETE) by the end of the simulation. Your internal briefing
document should cover the areas in which you intend to make specific
recommendations, e.g., elementary, secondary, tertiary institutions, open
and distance learning, etc., as well as the strategies you intend to use to
reach your goal.
II. The format and organisation of the individual sections
Each section addressing a substantive issue area should contain the
following information, presented in this format.
A. Identify the substantive issue.
1. What is the significance of the issue for the Eutropian Federation?
2. How is this issue important for your country or organisation?
279
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
B. Policy objectives.
1. What are the policy objectives of your country or organisation?
2. If these objectives cannot be reached, what are the secondary or
compromise objectives?
C. Negotiating strategies.
1. What are the various strategies that could be pursued in the
negotiations? A comprehensive paper will provide a variety of
strategies.
2. What are the advantages and disadvantages of each strategy?
D. Final recommendation
1. What is your team’s the final policy recommendation, in terms of
goals and strategies?
Guidelines for Writing the Policy Statement
The Policy Statement is in many ways a public version of the Internal
Briefing Document. In it, you state your country’s policies on the various
issues (what you favour and what you oppose), but you do not reveal
your negotiation strategies
NOTE to journalist teams: Present your Policy Statement in the form
of the inaugural issue of your newspaper, to be published on the first day
of the negotiations. In it, you introduce your publication, describe your
editorial stance and announce your publication schedule. Indicate how
you intend to cover the negotiations. Describe the relationship you would
like your publication to have with its readers. You might want to use the
opportunity to invite letters to the editor or other pieces of writing, for
example. Indicate who you hope will be interested in reading your
publication.
Technical consultant groups: Your Policy Statement is a chance to
advertise your company. Use this opportunity to profile your company
for the others. Tell them what kind(s) of technical expertise you can provide to the country teams as they negotiate a design plan for the Eutropolis
education system.
280
Appendicies
I. Overall Format and Organization of the Policy Statement
A. The Policy Statement is the product of a group effort. Therefore, it
should be published (i.e. sent to all other teams) in a single, unified
document rather than as a number of separate messages from individual
team members or from issue area working groups.
1. While the final document should be contained in a single file, it can be
written and revised in any number of separate sections in “Papers”
and then copied into a single message in the “Message Centre”. Send
this message to ALL teams, with the following subject line: “Policy
Statement.”
2. The format, i.e. margins, etc., of the document should be consistent
throughout.
B. The final Policy Statement should be as professional a document as
possible. It should contain the following components:
1. A table of contents.
2. An executive summary, or overview, which succinctly presents the
following:
a. The overall national policy goals.
b. The broad policy goals for each issue area.
C. Finally, the Policy Statement should not contain any policy
recommendations that are contradictory.
II. The Format and Organization of the Individual Sections
Each section should address a substantive issue area and should contain
the following information:
A. Identify the substantive issue.
1. As your delegation sees it, what is the significance of the issue for the
Eutropian Federation?
2. Explain how important this issue is for your country and why it is so
important.
B. How does your delegation propose to deal with or resolve this issue?
In addition, the position paper should include a bibliography listing any
references to secondary sources. The importance of including references
for quoted or paraphrased material cannot be overstated. It allows your
readers to check the original references and it demonstrates academic
integrity.
281
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Eutropia
282
Appendicies
Journalists
You are members of a team of journalists (Eutropian or national) who
publish an electronic newspaper or magazine. It is your job to report and
comment regularly on the progress of the Task Force’s negotiations as
well as on the work being done by the technical consultants.
Your first task is to organise yourselves as a team. This must be done
before the simulation starts. Choose a name for your publication and
decide how often you intend to publish. As soon as you have chosen a
name, send a message to the simulation moderator indicating what your
publication’s new name is. The moderator will then change your group’s
name in OPUSi.
Decide how you want to organise your team; your team’s tasks include
activities such as news reporting, editorial writing, doing background
research, editing articles and commentaries, interviewing members of
the Task Force on Education and Training, sifting through the existing
documents and communications for newsworthy stories. You can run your
organisation in any way you see fit. Remember, though, that the more
professional it is, the more seriously you will be taken by your readers.
Prepare a confidential statement outlining your publication’s editorial
stance (bias) and the strategies you intend to use to achieve your publishing goals. This document is the equivalent of the Internal Briefing
Document prepared by the country teams. Send a copy of this to the
simulation moderator.
Your second document, the equivalent of the country teams’ Policy
Statements, is the first (inaugural) edition of your publication. In it, you
announce the launching of your publication and introduce it to the other
participants in the simulation. Consider including some of the following
283
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
kinds of “market-relevant” information about your organisation: size, type
of publication, political or editorial orientation (bias), circulation (number
of readers). Consider “real world” models such as “The European,” “Le
Monde,” the “Chronicle of Higher Education,” CNN’s “Impact,” MSNBC’s
“Dateline” or the BBC’s “Broadcasting House.” Make your profile as
realistic as possible – this will add to your publication’s credibility.
Publish your inaugural issue on the Thursday prior to
the start of the negotiations (4 November).
Read the profiles of the various countries and consultants before the
negotiations begin. As soon as the Policy Statements have been published,
contact the other groups. Ask questions about their positions on various
issues, based on what they have said in their Policy Statements and what
you know about them from the team profiles. Make your questions specific,
and direct them to individual countries whose profiles suggest they might
have something interesting to say on that topic.
Once you have launched your publication, your next issue (you may
combine this with your inaugural issue) should be based on the country
profiles and the basic socio-economic and education-related data
contained in them. Your second issue might be based on the countries’
Policy Statements. If countries are late in sending their Statements, there
is nothing against your mentioning this in your publication, but bear in
mind that your success as journalists depends to a great extent on your
ability to establish and maintain good relationships with your readers and
with your information sources (contacts).
During the simulation, you will have the following
tasks to accomplish:
1. Produce an edition of your periodical at least twice a week. This will
contain at least two broad categories of text: (a) reporting on events
and developments and (b) editorial commentary on these or other
items and issues. Decide what your deadlines are – on what days and
by what time will you put out your periodical?
2. Regularly interview each country and technical consultant team to
obtain information and opinion on current issues.
284
Appendicies
3. When events warrant, publish a special issue covering a breaking
news story. You may also publish issues devoted to a particular
education-related theme or a particular aspect of the Task Force’s
work.
4. Invite your readers to contribute “op ed” pieces (views which disagree
with positions your publication has taken) and “letters to the editor.”
Make a clear distinction between reporting news (“facts,” events or others’
opinions) and offering editorial commentary (presenting your
organisation’s own opinion or point of view).
In addition to writing messages to the other groups, here are some actions
you can take: interviewing countries, organizing press conferences and
providing background information or documents related to the education
issues that are the focus of the simulation. Please do not “invent” major
facts or events without first checking with your facilitator. S/He will let
you know as quickly as possible if a fact or event is viable. Remember
that such things should be credible; otherwise they tend to reflect badly
on your organisation.
From time to time you may receive “leaked” information (sent to you by
others involved in the negotiations, for example) that you would not
normally have access to; you may also receive “wire service reports” or
“tips” that give you leads to follow up. Again, the credibility of your
periodical depends largely on the professional quality of your reporting
and commentary.
You might like to consider looks as well as content from a professional
angle. If you would like your publication to be Web-based, contact the
simulation moderator. Space will be provided on a Bremen-based server. Do not use a local server for this, as doing so would reveal your real
location.
Top
285
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
InfoTech Unlimited: Technical
Consultants
Highland
Government:
Highland is a parliamentary democracy.
Work:
Over half of the labour force (54.5%) works in the service sector (roughly
half of these in the public sector), another 30% work in manufacturing
and about 15.5% in agriculture. Unemployment has proved to be a
stubborn problem, and is currently around 18.5% nationally, with at least
one region experiencing levels as high as 26%.
Strict austerity programs in the early 1990’s allowed Highland to reduce
budget deficits and government indebtedness to acceptable levels so Highland could join the monetary union, and budgetary discipline has kept the
national debt at just under 60% of the gross domestic product since that time.
Government pressure has also kept wages lower than elsewhere in the
Federation, which might have attracted new investment if the infrastructure
had been adequate. Instead, by the turn of the century Highland’s gross
domestic product was still less than 70% of the Federation average per
capita. Not surprisingly, Highland receives more money from the
Federation in the form of subsidies than it contributes; 8% of the national
budget is currently supplied by the Federation.
At the end of the 1990’s, in an attempt to reduce Highland’s dependence
on Federation funds by revitalizing the Highland economy, the government began to increase spending to improve the country’s infrastructure
and to renew its industrial base.
286
Appendicies
The campaign to make Highland a more attractive location for new
investment included a program of incentives to encourage the use of
information technologies in the workplace. However, Highlanders initially
resisted, fearing that further computerisation would cause even more
unemployment. Currently, there are only 56 computers per 100
employees - the lowest ratio in all Eutropia.
Society:
Highland has a total population of 9,983,050. The overwhelming majority
of Highlanders are Catholic; with the exception of a small community of
citizens of African descent (less than 5%) who moved to Highland during
decolonization the population is also ethnically homogeneous.
Highlanders, as a rule, identify more strongly with the local district in
which they live than with Highland’s national government, and district
governments have more autonomy than is the case in most Eutropian
member countries. On some issues, Highland behaves like a federation
within the Eutropian Federation; responses to social problems such as
crime or substance abuse frequently have a distinctly local character.
Education System:
Comprehensive schooling is compulsory until the age of 16, after which
apprenticeships or, alternatively, vocational or technical training are
available for those who do not complete the final two years of
comprehensive schooling to qualify for university.
Subjects taught and teaching methods used are prescribed by provincial
culture and education ministries and there has been little encouragement
or tolerance of innovation in teaching. Attempts to increase the national
government’s influence on education policy-making and to set national
performance standards have met with limited success.
Because of this, wealthy Highlanders tend to send their children abroad
for their secondary and tertiary education. A significant number of these
later decide to live and work as expatriates in other countries, leading to
a serious “brain drain” problem in Highland.
287
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Classical languages are required of students preparing to study at university, and bilingual education has been introduced on a limited basis in
the border districts, but foreign languages are otherwise optional and
offered only in the last four years of the school curriculum. Hokish is the
official language of instruction.
Highland is home to some of Euturopia’s oldest universities.
In the 1980’s and 90’s, as Highland struggled to meet Federation criteria
for joining the monetary union, Highland’s tradition-rich educational system paid a heavy price. Austerity measures and budget restrictions
repeatedly led to severe cutbacks in the already modest resources available
for education. Teachers and professors alike were poorly paid and
demoralized, equipment was hard to come by and buildings were poorly
maintained.
Today the situation is showing signs of improving as the economy improves.
Class sizes are still intolerably large at all levels, particularly at universities,
library holdings are outdated, and computer equipment is in short supply,
but the government has finally understood that competitiveness in
Eutropian and global markets requires a well-educated, technology-literate workforce. As a result, Highland has begun to invest in facilities and
information technology. In order to have more money available for national education initiatives, Highland’s government wants to keep spending
for the Eutropolis education system at an absolute minimum.
On the other hand, the Government has had to acknowledge the existence
of a seemingly insoluble problem regarding finance: as standards in
schools improve, more people are seeking admission to tertiary
institutions, putting an immense strain on the current practice of funding
for all who gain a place in a tertiary establishment - and this at a time
when more investment in equipment and personnel is being demanded.
One proposed solution is to make tertiary education available to as many
as possible, as cheaply as possible, via an “Open University,” with material transmitted via television, radio and the Internet, and making considerable use of teleconferences. Two problems arising from this are the
question of entrance requirements and whether qualifications won in such
a way would be fully accepted by industry and employers as the equivalent
of more traditional qualifications.
288
Appendicies
Literacy:
85% of the population over the age of 15 can read and write; this is the
lowest literacy rate in Eutropia. Literacy rates are significantly higher for
males (90%) than for females (81%). The discrepancy is even more
noticeable when it comes to computer literacy; 55% of Highland’s male
population between the ages of 7 and 45 is computer and Internet literate, compared to 38% of the female population. One hopeful sign:
Highlanders under 25 years of age accept and use computer technology
at a much higher rate than the national statistical average.
Electronic Information Infrastructure:
Highland has a generally adequate integrated network of coaxial cables,
open wire, microwave radio relay, satellite earth stations and submarine
cables to off-shore islands. Highland’s mountainous terrain means
improvements to the infrastructure are costly and technically challenging.
289
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix F:
Assignment for designing an evaluation of IDEELS
Contemporary Perspectives on Research in Pedagogical Information Technology
Research Seminar - Fall 1998
Semester Assignment – 3
In the semester oppgave you will evaluate your chosen scenario from
one of the perspectives covered in the class this semester: psychometrics,
distributed cognition, activity theory, discourse analysis. This perspective
gives you boundries for the type of research question you propose (problem stillingen) the type of data you need to collect, how you collect it
and how to analyse it. When coming up with a research question, you
could begin by discussing among yourselves the types of data you can
collect and how it can be used (e.g., structured interviews, participant
observations, video taping etc.) and the types of questions you can ask of
the data. What I am trying to say is that do not just think of it from the top
down (question —> data collection etc), rather think ALSO from a bottom up (this data —> what questions).
When it comes to what to include in the written report, you should have
sections such as:
1. the scenario (describe the scenario, the learning environment including
the actors involved (i.e., students, teachers, facilitators, technical helpers, tutors, the school or university, parents, rectors etc.), the artefacts (i.e., the actual CSILE system, the technologies in IDEELS - email,
teamwave, OPUSi, the internet, your own editor etc. -, the OthoDL
290
Appendicies
database etc.) and so on. When you start writing this part you will
probably begin to get some ideas as to the types of data you can
collect (created artefacts, electronic data logs of email, interviews with
people, videotapes of people using the system etc..)).
2. the theoretical perspective (here you will demonstrate that you understand the main aspects of the perspective - describe it in general, its
theoretical roots, the types of research carried out here etc., where
the theory focuses (e.g., distributed cognition focuses on people
interacting through artefacts) etc.
3. the research question you will ask. Motivate why you want to ask this
question.
4. Data to be collected - describe the data you need and how you will get
it. If you will have a questionnaire, make up the questionnaire, if a
structured interview design the questions, if an unstructured interview
describe what topics you hope to get out of it and a few leading
questions you will use etc.
5. How will you analyse the data to answer the question you have asked.
Again, for the report length, the content is more important - that you
include the above points is a minimum. Try to keep the report between
10 and 20 pages. Hand in 3 copies of the report
On the 1st exam day the group will present their projects to the entire
class. This is part of the exam. There will be allocated 15 minutes per
group.
291
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix G:
Interview Guide for IDEELS (English)
Interview #:
Date:
Interviewee:
General
• How has the assignment work and the group functioned?
Division of labour
• Who did what?
* Why?
* How?
• How did you arrive at the particular division of labour?
* Process?
* (example, responsibility for webpages)
• Did everyone agree on the chosen division of labour?
* division of workload
Communication
• What communication forms did you choose
* Face-to-face, computer mediated, telephone
* What computer tools did you use? Why?
* General on TW as a communications tool (use / lack of use)
Integration
• How did the assignment work function in relation to your other
everyday tasks?
• How did this effect the assignment work/group collaboration?
* in particular: division of labour and forms of communication
Goal Setting
• What did you understand the goals of IDEELS to be?
* generally and specifically
292
Appendicies
Intervjuguiden (Norsk)
Interview #:
Date:
Informant:
Generelt
• Hvordan har arbeidet med oppgaven og gruppen fungert?
Arbeidsdeling
• Hvem har gjort hva (hvilke oppgaver bestod arbeidet av? hvorfor?)
• Hvordan kom dere frem til den arbeidsdelingen? Prosess. (eks webside)
• Var alle enige i den arbeidsdelingen som ble valgt? (fordeling av arbeidsmengde)
Kommunikasjon
• Hvilke former for kommunikasjon valgte dere? Hvorfor? (face-to-face,
telefon, datamediert?)
• Hvilke verktøy brukte dere? Hvorfor?
• Kan du si litt om teamwave? (bruk / mangel på bruk)
Integrasjon
• Hvordan fungerte det å arbeide med denne oppgaven i forhold til
resten av dine daglige gjøremål?
• Hvordan påvirket dette samarbeidet og valg av arbeidsdeling og kommunikasjonsformer?
Målsetting
• Hva oppfattet du som målsettingen i å være med i IDEELS? (generelt
og spesifikt)
293
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix H:
Pre-simulation survey for IDEELS
294
Appendicies
295
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
296
Appendicies
297
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
298
Appendicies
Appendix I:
Post-simulation survey for IDEELS
299
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
300
Appendicies
301
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
302
Appendicies
Appendix J:
Planning Tables from the Instructor’s View (Phases II-IV)
II Training
Task
Date
Giving instructions 18.02
for training
Data source State of source
Comments
Responsible
Instructions Prepared
available in TW
Helge
Put out files at
pre-specified
times
24.02- Files (Expert Prepared
5.03
Rankings,
Permission on
Activity 4 room)
Helge
Follow training
(general)
24.02- T W
5.03
Follow training
activities
24.025.03
Helge
Activity 1
24.02 - Activity 1 room Prepared
3.03
Helge
Activity 2
24.02 - Activity 2 room Prepared
3.03
Helge
Activity 3
24.02 - Activity 3 room Prepared
3.03
Helge
Activity 4
3.025.03
Receiving
Assistance
Requests
students
24.02- E-mail from
5.03
students
*Agreed upon Helge
method for
answering
Distributing
assistance?’s if
necessary
24.02- E-mail from
5.03
students
Send forward Helge >
if cannot
Barbara or
answer!
Trond > *
Answering
content ?s
24.02- E-mail from
5.03
Helge
Barbara
Answering
technical ?s
24.02- E-mail from
5.03
Helge
Trond
Activity 4 room Prepared
Maintain message 24.02- Message
boards with ?’s
5.03
boards in
Help Room
Keep server
running
Just to seen
in general
Prepared
24.02 - TW server on
5.03
grevling.ifi.uib.no
Available
from 3.02
Barbara, Lars,
Arnstein
Helge
*Need routine Helge (w/
for checking Barbara
message
& Trond)
boards
Trond
> “drift”
303
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
III Designing an instructional room
Task
Date
Data source State of source
Comments
Following up the
design of an
instructional room
Responsible
All researchers
Receiving
Assistance
Requests
students
24.02- E-mail from
5.03
students
*Agreed upon Helge
method for
answering
Distributing
assistance ?’s
if necessary
24.02- E-mail from
5.03
students
Send forward Helge >
if cannot
Barbara or
answer!
Trond > *
Answering
content ?s
24.02- E-mail from
5.03
Helge
Barbara
Answering
technical ?s
24.02- E-mail from
5.03
Helge
Trond
Maintain message 24.02- Message
boards with ?’s
5.03
boards in
Help Room
Keep server
running
Prepared
*Need routine Helge (w/
for checking Barbara &
message
Trond)
boards
24.02- TW server on
5.03
grevling.ifi.uib.no
Trond > “drift”
IV Follow-up
Task
Date
Data source State of source
Comments
Responsible
Receive course
assignment
12.04
Individual
student
assignments
Each
instructor
responsible
for receiving
their own
student’s
assignments
Barbara,
Arnstein, Lars
Received
Tell class about the To Be
research in DoCTA Decided
Barbara
Take down
TW server
Trond
304
To Be
Decided
Appendicies
Appendix K:
Planning Tables from the Researcher’s View (Phases I-III)
I Preparation
Task
Data source State of source
Comments
Prepare computer 01.02
supported
collaborative
learning
assignment
Date
Assignment file Prepared
(merge
together
activities)
Part 2 of
Barbara
assignment is
VisArt
Prepare profile
questionnaire
15.02
Profile
Prepared
questionnaire
On web (make Kurt
available 24.02)
Prepare
15.02
pre-questionnaire
PrePrepared
questionnaire
On web (make Kurt
available 7.03)
Prepare postquestionnaire
15.02
PostPrepared
questionnaire
On web (make Kurt
available 25.03)
Prepare student
consent form
15.02
Consent form Prepared
Send to Lars Barbara > Lars
and Arnstein & Arnstein
Getting students’ By
consent
24.02
Consent form Prepared
Receive from Barbara
Lars and
Arnstein as well
Assign
researchers to
a team
Team list
21.02
Getting
students’ profile
Prepare for
video taping
Prepare for
interviews
Prepared
Profile
Prepared
questionnaire
file
24.02
Responsible
Barbara >
researchers
student >
Kurt
Eskil & Knut
Each
researcher
305
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
II Training
Task
Date
Follow what is
going on
25.02- T W
7.03
Data source State of source
Comments
Prepared
Getting the self- 25.02- Email to Helge Prepared
evaluation results 7.03
Interviewing
students
25.02- TW, emails
7.03
Responsible
All
researchers
Helge
According to Each
interviewing Researcher
needs
Collecting
assistance
requests
Email from
students
Helge w/
Barbara &
Trond
Getting logs
TW
TW >Øyvind,
Getting versions
control
TW
TW >Øyvind,
Selecting teams
for following
7.03
Data logs for
phase 2
Select teams Øyvind
to follow in
phase 3
III Designing an instructional room
Task
Date
Data source State of source
Follow what is
going on
8.0326.03
TW
Comments
Prepared
Responsible
All researchers
Getting the self- 8.03evaluation results 26.03
Email to Helge
Make available Helge
to other
researchers
Interviewing
students
8.0326.03
TW, emails
According to Each
interviewing Researcher
needs
Collecting
assistance
requests
8.0326.03
Email from
students
Make available Helge w/
to other
Barbara &
researchers Trond
Getting logs
8.03 26.03
TW
Make available
to other
researchers TW >Øyvind,
Getting versions
control
8.03 26.03
TW
Make available
to other
researchers TW >Øyvind,
Collecting chat
messages
8.0326.03
Emailed
Chat file
Remind
students to
send chats
306
Each
researcher
Appendicies
Appendix L:
Planning Tables from the Student’s View (Phases I-III)
I Preparation
Activity
Date
Documents/
instruments
State of Source
Comments
Responsible
Sign an agreement 16.02
to participate in
research
Consent
agreement
Prepared
Paper version Barbara >
instructors >
students
Get TW account
24.03
List of accounts Prepared
Email list to all
participants
Barbara >
students
Get team list
24.03
List of teams
Prepared
Email list to all
participants
Barbara >
students
Getting instructions 16.02
Instructions
(Explaining 3
phases, their
purpose, the
deadlines, the
assistance
provided etc.)
Assignment
document
completed &
delivered
Electronic form Barbara >
of VisArt part in instructors >
Classroom
student >
Completing profile 25.02
questionnaire
Profile
Prepared
questionnaire
Electronic form student >
on web;
Kurt (collects,
e-mailed auto puts in file)
matically to Kurt
307
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
II Training
Activity
Date
Getting instructions 25.02
for training
Documents/
instruments
State of Source
Instructions for Prepared
the training
session
Comments
Responsible
In TW training Helge >
room
student
Doing training
activities
25.02 - Training documents
5.03
Prepared
Activity 1
25.02- Training
4.03
document
Prepared
In TW activity
1 room
Helge
Activity 2
25.02- Training
4.03
document
Prepared
In TW activity
2 room
Helge
Activity 3
25.02 - Training
4.03
document
Prepared
In TW activity Helge
3 room
Activity 4
4.03 –5.03
Training
document
Prepared
room
In TW activity 1
Helge
Prepared
on web;
e-mailed
automatically
to Helge
Electronic form
student >
Helge
Completing the
self-evaluation
tool
Self-evaluation
test
Participate in
an interview
5.03
Getting
assistance
25.02- Help and
5.03
resources
308
Interview
Guide
Helge
TO DO
According to Each
each
researcher
researchers
needs
TO DO
Help rooms; Student >
Email > Helge Helge >
Barbara or
Trond > Student
Appendicies
III Designing an instructional room
Activity
Date
Documents/
instruments
Completing prequestionnaire
8.03
Designing an
instructional
room
Getting
assistance
Comments
Responsible
PrePrepared
questionnaire
Electronic
form on web;
e-mailed
automatically
to Kurt
Student >
Kurt (sort
questions) >
Interested
researchers
8.0327.03
Assignment
Document
Paper copy
and in TW
Classroom
Barbara
8.0327.03
Help Room
Prepared
and resources;
Assistance plan
TW Help Room; Student >
Assistance plan Helge >
ready
Barb or Trond >
Student
Completing post- 26.03
questionnaire
Postquestionnaire
Electronic form
on web;
e-mailed
automatically
to Kurt
Participating in
an interview
Interview
guides
8.0327.03
State of Source
Prepared
TO DO
Student >
Kurt (sort
questions) >
Interested
researchers
Selected
Responsibility
students only of individual
as needed
researchers
309
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix M:
VisArt Consent Form
Student Consent to Participate in Research
A research team headed by Dr. Barbara Wasson is currently conducting
a research project DoCTA: Design and use Of Collaborative Telelearning
Artefacts, that is investigating the design and use of telelearning activities with distributed students. The institutes involved in the research include:
the University of Bergen, Stord/Haugesund College and Nord-Trøndelag College and Telenor FOU (Kjeller). Our intention is to gain a better
understanding of student learning in this new mode of course delivery,
more specifically collaborative learning over the internet with a distributed
team of students. As part of a course at each of the teaching institutions,
a collaborative telelearning activity, called VISART, has been designed
to have teams comprised of a student from each of the teaching institutions
collaborate through an internet-based groupware system called TW.
Through your responses to questionnaires, interviews and through
electronic data collection we hope to find more about your opinions of
the software, the activities, collaboration, telelearning in general, etc.
There are 8 graduate student projects associated with VisArt and we
hope that you will agree to provide them with enough data that they can
carry out their projects.
The activity has been designed with a preparation phase (installing software, etc.), a one-week training phase and a three-week activity phase.
During each phase you will be asked to fill in questionnaires, perhaps
participate in an interview, and agree to have your team email (you will
have a special team account to use for project planning, etc. that will go
to a researcher as well, any private email can be sent using your usual
email account and WILL NOT be collected) and team electronic chats (to
310
Appendicies
be saved by you and sent to a researcher) collected. All information
collected will be kept strictly confidential and will be used for research
purposes only. Publication of results and final reports will in no way identify
individual students and your person number will not be collected. This
research is solely for the advancement of knowledge about the conditions
under which people learning given this new technological media. There
are no other motives. The data collected will in no way inform your course
grade. In addition, the project is registered with NSD and their regulations
for data collection, storage and deletion are being followed.
If you are willing to participate in this study, please sign this consent form
and return it to your instructor.
Your cooperation in the collection of this information is greatly
appreciated, but is completely voluntary. You have the right to choose
not to participate in this research project. Your decision will in no
way affect your grade.
Please indicate by checking in one of the boxes below whether you freely
consent and agree to participate in data collection for research in
collaborative telelearning. If you have any questions regarding these
procedures, please feel free to contact Barbara Wasson at 55 58 41 20
or by email at [email protected].
The research is funded by KUF’s IT I utdanning programme.
•
yes, I do consent to participate
•
no, I do not consent to participate
Name & Institute (please print):_________________________________
Signature:_______________________ Date:________________
311
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix N:
Semester Assignment
IVP 532, Spring 1999
Assigment 1: Computer Supported Collaborative
Telelearning
This is both an individual and collaborative assignment. It is individual in
that each student will hand in a report and receive an individual mark. To
write the report, however, entails participating in a collaborative telelearning
activity called VisArt. VisArt is part of the DoCTA (Design and use of
Collaborative Telelearning Artefacts) research project which is focused on
computer supported collaborative learning (CSCL). DoCTA is lead and
coordinated by Barbara Wasson and the VisArt scenario is a collaboration
between the University of Bergen (UiB), Høgskolen I Nord-Trøndelag (HiNT)
and Høgskolen Stord/Haugesund (Stord). During the collaborative activity
you will be assigned to a team consisting of 3 students, one from each of
UiB, HiNT and Stord. The team will collaborate through some specially
tailored groupware called TeamWave Workplace. You should have turned in your consent form before starting this assignment so we know
whether you and your team will contribute to the data collection.
The assignment has 3 main parts:
1. Read the background literature on CSCL and awareness.
2. Participate in the VisArt Collaborative Activity.
3. Write an individual report that reflects on the process from both a
practical and theoretical perspective (details of the report to follow).
1. Literature on CSCL
The following literature should be read:
Dillenbourg, P., Baker, M., Blaye, A. & O’Malley, C. (1995). The
Evolution of Research on Collaborative Learning. In P. Reimann H.
Spada (Eds). Learning in humans and machines. Towards an
interdisciplinary learning science, 189- 211. London: Pergamon.
312
Appendicies
Koschmann, T. (1997). Paradigm shifts and instructional technology:
An introduction. In T. Koschmann (Ed.) CSCL: Theory and practice
of an emerging paradigm, 1-23. Mahwah, NJ: Lawrence Erlbaum
Associates.
Salomon, G. (1992). What does the design of effective CSCL require
and how do we study its effects? SIGCUE Outlook, Special Issue on
CSCL, 21(3), 62-68.
Gutwin, Carl; Stark, Gwen; Greenberg, Saul: Support for Workspace
Awareness in Educational Groupware. Proceedings of CSCL’95.
Mahwah, NJ: Lawrence Erlbaum Associates, Inc. http://wwwcscl95.indiana.edu/cscl95/toc.html.
Some pointers to CSCL on the web:
PedInfo ’98: http://www.ifi.uib.no/staff/barbara/courses/pivar98a2projects.html
Outlook: http://www-cscl95.indiana.edu/cscl95/outlook/
Outlook.Contents.html
CSCL’95: http://www-cscl95.indiana.edu/cscl95/fullprg.html
CSCL’97: http://www.oise.utoronto.ca/cscl (click on plenary sessions
& parallel sessions)
Collaboration Projects: http://www.hypernews.org/HyperNews/
get/www/collaboration.html
2. VisArt Collaborative Activity
Objectives:
• To provide an opportunity for participation in a distributed collaborative
learning activity where team members do not have the opportunity to
meet face-to-face. This will give experience in both distributed
collaboration and in the use of a groupware system over the Internet.
It will hopefully highlight both positive and negative aspects.
· If each team member has given consent to participating in the research
aspects of this scenario, your team’s experience will also contribute
to furthering our understanding of this new learning phenomenon.
313
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
In the VisArt activity you will:
• Be part of a group of 3 (see list: 1 Bergen, 1 HiNT, 1 Stord) who will
form a design team (each team will be assigned a number). This same
number will be used to identify the team’s working room in TeamWave
Workplace (e.g., team 1 will use the Team1 room) and the team’s
common email address (e.g., team 1 will have [email protected]).
• Organise a collaborative team effort
• thinking Salomon’s definition genuine interdependence: 1) sharing
information 2) division of labour 3) joint thinking
• Carry out the Design Activity with the help of:
• help pages (http://www.ifi.uib.no/docta/VisArt/help)
• assistance (email [email protected])
• Help Room
• Produce 2 items:
• A document of your pedagogical decisions (e.g., who is the room
intended for, what is the content, etc….)
• A TeamWave Workplace room for teaching/learning
The VisArt activity consists of 5 major parts:
VisArt Activity
Date(s)
Preparation
Week 8 (from February 25)
Training on Tools & Collaboration
Week 9 (March 1-5)
Pre-questionnaire
March 7
Design Activity
Week 10-12 (March 8-26)
Post-questionnaire
by March 26
Preparation:
In preparation for VisArt you will need to download TeamWave Workplace
and some extension files that you need. Directions for this preparation
can be found at http://www.ifi.uib.no/docta/teamwave/. Use the logon
id that your instructor has given you. Your team’s email should be working
as well. If you experience problems, email [email protected].
If you have not delivered your consent form to your instructor please do
so. If you have agreed to participate in the research, you will need to fill
in a participant’s profile questionnaire that can be found at http://
www.ifi.uib.no/docta/VisArt/profile.html.
314
Appendicies
Training:
During the training session you will be taken through a number of activities where you will gain experience in using the technological tools and
gain experience in collaborating while at the same time getting to know
your team members. A self-evaluation questionnaire will be provided
to help you gauge your competence in using the tools. All training will
take place in TeamWave Workplace Training Room and you will get
more information about this week 8.
Pre-questionnaire:
The pre-questionnaire that asks about your attitude towards TeamWave
Workplace so far needs to be filled in. The questionnaire is can be found
from March 5 th at http://www.ifi.uib.no/docta/VisArt/prequestionnaire.html . Please fill it before March 8th.
Design Activity:
In the VisArt design activity your team is to design a TeamWave Workplace
room for learning about some topic. We are most interested in your
process of collaborating – how did you make your decisions and carry
out them out. Therefore, the outcome of this activity will include the
learning room as well as documentation about major pedagogical decisions
that you made. More details about the design task follow:
1. The room you chose to design should enable the students to know
more about a concept, a procedure, a theory, a process, etc. Some
examples of possible topics include a room for:
• 6th grade science (e.g., birds, dinosaurs, pollution, etc.)
• high school geometry
• a university course on programming (e.g., recursion, file structures, etc.)
• a theme in an art history course
• OR a topic approved by the activity leader
([email protected])
2. The room could contain a presentation, a game, a tutorial, a
questionnaire, an exercise, or some combination of these and utilise a
number of the TeamWave Workplace tools. The room you develop
should be under your team’s working room.
315
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
3. When documenting the pedagogical decisions that your team makes, you
should include issues such as characteristics of the student audience (e.g.,
age, grade, etc.) and the complexity of the subject area. Create a file PedDecteamx (i.e., use the fileholder tool and substitute your team number in
teamx – team 1 is PedDec-team1) and leave the file in your working room.
Post-questionnaire:
The post-questionnaire that asks about your attitude towards the training, telelearning and TeamWave Workplace after the VisArt has been
completed needs to be filled in. It can be found after March 25th at
http://www.ifi.uib.no/docta/VisArt/post-questionnaire.html.
3. Individual Report
The individual report is the document you will hand in to be counted
towards your grade for the semester assignments (i.e., it will count ½ of
the semester assignment grade). The individual report should contain:
• an introduction to computer supported collaborative learning and
collaborative telelearning (use literature in class + other information
you might find on the web, in the library etc.)
• a description of the design activity you were asked to participate in,
including the tools provided and used (e.g., e-mail, TeamWave
Workplace, etc…)
• a presentation of your team’s room (maybe take a screen snapshot
here) and the pedagogical decisions you made to get to the solution
• a discussion of how the team met Salomon’s requirements for genuine
interdependence and whether or not TeamWave Workplace
supported activities resulting from attempts at meeting Salomon’s
requirements
• a discussion of Gutwin et al.’s awareness concept and what it means
in conjunction with your distributed collaboration through TeamWave
– does TeamWave Workplace support awareness? How?
• your general reaction to collaborative telelearning (as you experienced
it) including: a reflection on the team’s work focusing on your participation
and the others’ participation (did you feel you did more work, someone
did not pull their weight etc. – such comments will not work against
them (you do not need to give names) and are private to the instructor),
the process of carrying out the assignment, general comments about
the entire assignment, your reaction to TeamWave Workplace (did it
help or hinder your work, was it easy to use, hard to use, etc.)
316
Appendicies
Appendix O:
Directions for TeamWave Workplace Installation
317
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix P:
Designed Learning Rooms from VisArt
Below you will find a number of examples of the rooms designed in the
VisArt scenario.
Astronomy:
318
Appendicies
Threatened Species:
Renewable Energy:
319
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Gothic Architecture:
Publishing on the Internet:
320
Appendicies
Triangles and Pythagoras:
The Big Bang:
321
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Propaganda:
322
Appendicies
Appendix Q:
VisArt Profile Questionnaire (Norsk)
323
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
324
Appendicies
325
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
326
Appendicies
327
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix R:
VisArt Self-Evaluation Questionnaire
328
Appendicies
329
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
330
Appendicies
Appendix S:
VisArt Pre-questionnaire
331
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
332
Appendicies
333
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
334
Appendicies
335
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
336
Appendicies
337
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
338
Appendicies
339
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix T:
VisArt Post-questionnaire (Norsk)
340
Appendicies
341
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
342
Appendicies
343
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
344
Appendicies
345
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
346
Appendicies
347
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
348
Appendicies
349
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
350
Appendicies
351
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
352
Appendicies
353
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix U:
Norweign Version of Training Assignments 1,3 & 4
(Norsk)
Oppgave 1
Velkommen til din første TeamWave oppgave.
Denne oppgaven består i at hver person i gruppen skal intervjue en av
de andre gruppemedlemmene (f.eks. A intervjuer B som intervjuer C
som intervjuer A). Du skal samle informasjon og deretter presentere
dette for den 3. personen i gruppen, (f.eks. A skriver en presentasjon av
B, B skriver en presentasjon av C og C skriver en presentasjon av A).
Forslag til gjennomføring:
• Gå til rommet tilhørende din gruppe.
• Bruk ”Calendar” for å planlegge tid for intervju.
• Bruk ”Chat” for å gjøre intervjuet.
• Lag en ”To-do list” for gruppen din.
• Presenter intervjuet til de andre i gruppen din ved hjelp av en ”File
Viewer”.
• Lag ditt eget rom (Room, New Room). Dette kaller du det samme som
brukernavnet ditt. For å begrense innsyn til rommet ditt går du til
Permissions og ”setter” at bare din gruppe kan gå inn i dette rommet.
• Lag en ”Doorway” tilbake til klasserommet fra ditt personlige rom.
Når du og gruppen din er ferdig med oppgave 1, kan dere begynne på
oppgave 2. Dersom du skulle få tid til overs mens dere holder på med
oppgave 1 eller 2, kan du begynne på oppgave 3, som er en individuell
oppgave.
354
Appendicies
Oppgave 3
Disse oppgavene løser du i ditt personlige rom, dersom ikke annet blir
anbefalt.
Verktøy
Oppgave
Web browser
Hent en web-side inn i TeamWave Workplace.
Prøv hvordan denne nett-losen fungerer.
URLRef.
Bruk ”Help”-funksjonen og finn adressen til
TeamWave sin hjemmeside.
Legg så denne inn i en URLRef. og prøv den.
Post-it
Fortell dine gruppemedlemmer hva du synes om
TeamWave så langt ved hjelp av en post-it lapp
(i grupperommet).
Vote tool
Dersom du ikke har gjort det ennå, avlegg din
stemme i “Classroom”.
Meeting roster
Planlegg det du vil anse som å være en/et
spennende debatt/møte mellom noen av Norges
politikere, du bestemmer tema/mål for møtet.
Database
Lag en enkel database for cd-samlingen din (tre
felter, f.eks. Navn, Tittel, Nasjonalitet).
Legg in 5 album i databasen.
Eksperimenter med ulike typer felter.
Image whiteboard
Tegn huset ditt!
Prøv også å importere et bilde, og se hvordan du
kan kombinere dette.
Page
Fortell dine gruppemedlemmer at du er ferdig med
Oppgave 3
Oppgave 4
I oppgave 4 skal dere sammen finne frem til et hvilket som helst tema
som dere skal presentere for de andre gruppene ved å tilby en eller
flere linker til sider som omhandler det valgte tema.
Forslag til fremgangsmåte:
• Foreslå og bli enig om tema, ”brainstorming”, ca. 30 min.
• På egenhånd finne sider på www som inneholder informasjon om det
tema dere har valgt. Dette kan gjøres til neste dag.
• I fellesskap finne frem til hvilke linker som bør benyttes, ca. 1-2 timer.
è Gi en kort presentasjon av deres tema ved hjelp av linker til de sidene
dere ble enige om.
355
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Denne arbeidsprosessen følger de krav som G. Salomon setter til ”genuine
interdependence”.
1. dele informasjon, (the need to share information )
2. arbeidsdeling, (division of labor among the team members)
3. felles tenkning, (pooling together of minds / joint activity of thinking)
Det er G. Salomons oppfatning at for å lykkes med CSCL, (Computer Supported Collaborative learning) må “interdependency”
være tilstede.
For det videre arbeid i VisArt anbefales det at dere i størst mulig grad
prøver å arbeide sammen på denne måten.
356
Appendicies
Appendix V:
The Narg Island Game
Narg Island53
53
Reworked and Developed from AVIAT’s Caribbean Island Survival II Exercise
357
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
14.45 HRS on Sunday 5th of June.
Sailing on the 15metre yacht ‘The
Pel’ in the centre of
a 10-km wide
stretch of one of the
most beautiful and
remote Norwegian
Fjords. The boat is
close to the southern
tip of Narg Island,
which is a bare
expanse of rock located approximately in the middle of the wide Fjord.
Bjørn, the skipper is shouting to you over the noise of a sudden severe storm
Here take the wheel and keep her away from the rocks”
Suddenly the wheel goes loose in your hands. The rudder has snapped!
“Oh No! There goes the rigging…. “
“Loose the sail before she breaks!”
Over the sound of the storm everyone hears the noise of the mast breaking
“Stand Clear everyone…”
“Push the mast out overboard before it knocks anyone out!”
“Someone get below and start the engines.”
Nils-Erik emerges from below looking worried
“The engine compartment is completely flooded and we are taking
on water fast”
“The radio is dead. So are all the electrics!”
358
Appendicies
“Quick get your lifejackets on. Abandon ship!”
The rudderless ship is now at the mercy of the storm, with 3 to 4 metre
waves and 70 km/hr winds, which are pushing you towards the cruel
rocks of Narg Island.
Bjørn, the skipper shouts:
“Stand by… we are going to hit the island!”
“Everyone jump overboard and head for the shore!”
15.03 HRS on Sunday 5th of June.
As Bjørn predicted the boat hits and is smashed into the rocks of Narg
Island. The cruel waves destroy everything. Even the small inflatable lifeboat
is destroyed. The boat sinks rapidly into the extreme depths of the Fjord.
Thankfully you manage to scramble up the rocks in the raging storm.
“Help Me. “ cries Solveig as she is covered in blood from cuts over
her arms and legs.
“Quick lets take cover out of the storm in the sheltered cove on the
other side of these rocks!”
“My God! This is the worst storm I’ve ever seen!”
“Look! There is Bjørn floating in the water. Quick help me get him
up out of the water!”
“Does anyone know first aid?”
“He’s out cold. Quick stand back! I’ll have to try mouth to mouth!”
“Look at his arm… its bent over double and black.”
“That looks bad.”
“Does anyone have any ideas what we can do?”
359
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
15.12 HRS on Sunday 5th of June.
All of your group are out of the raging seas and are sheltering from the
worst of the storm in a small cove. Your leader, Bjørn, looks in bad shape.
After mouth to mouth resuscitation he has started breathing but his arm
is obviously badly broken and is swelling. He has not regained
consciousness and seems to be having a fever. The rest of you are in
better shape with just some deep cuts and feeling very wet and cold.
Before “The Pel” went to its watery grave you each managed to grab
some items from the boat and have added some other things from the
wreckage remains that smashed against the rocks after the boat went
down (a list of the items are given on page 8)
You wanted an adventure holiday, but not like this! It has been 7 days since you
rented the boat from Bergen harbour for a 14-day bare-boat charter in the fjords.
Bare-boating is chartering a boat without a professional skipper and crew.
Since leaving Bergen, ‘The Pel’ has
visited several points along the coast
always heading north for the remote
beauty in that area. Most recently
you sailed around the northern tip
of Hammerfest and spent last night
in the small town of Børselv.
This morning, all of you spent two
hours scuba diving in dry suits off
Maelstrom Rock near Narg Island.
Suddenly, a violent storm came up,
causing enough concern for Bjørn to
decide to weigh anchor and head for
the nearest harbour (uninhabited) 30 km further at the far end of the fjord.
The cove in which you are sheltering is about 400 metres from the rocks
where the boat was lost. The water bordering the island is very deep
(500m); the cliffs and rocks around the edge of the island are bare and
dangerous. They range in height from 200 to 300 metres. The beach is
about 30 metres long and 8 metres deep, rising sharply into rocky and
rugged terrain. The only inhabitants of the islands are the colonies of
huge gulls on the top of some of the higher cliffs.
360
Appendicies
Narg Island is a very remote place. Only occasionally does a cruise or
any other boat visit this end of the fjord, which is uninhabited and somewhat
foreboding.
Sometimes on summer weekends Maelstorm Rock will attract some of
the more experienced scuba divers that want a deep dive in icy waters.
Narg Island is mountainous with sheer cliffs surrounding most of the
perimeter of the island, rising from the sea to about 200 metres. It is a
very rocky and rugged terrain with little vegetation. There are several
small scrubby trees that tell of very severe weather. Animal life is limited
to a large colony of gulls and the odd seal which suns itself on the low
beaches during warm weather. There is no fresh water on the island.
Air temperature in the area for June is typically 15-22 degrees centigrade
during the day and 7-10 degrees centigrade at night with 12 to 18 knot
winds. The relative humidity is about 60 to 65 per cent. During storms,
the temperature is known to suddenly drop some 20 degrees or more. In
the very worst summer weather snow is not unknown!!
Water temperature in June is 10 to 13 degrees. There is some pretty
361
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
constant fog in this part of the fjord, and this time of the year brings
some heavy rainfall and brief storms, which may last up to 48 hours.
Sunrise is 3.10 a.m. and sunset is 23.15 p.m.
“Jan, what possessed Bjørn to choose this place?”
“Simple. The island provides some shelter from the storm. When
Bjørn realised he would not make it to the end of the fjord he hoped
we could hide from the worst of the storm behind the island.”
“At least we are all out of the water!”
“Yes that is true. Now we need to decide what we must do next. Any
suggestions?”
“I wish Bjørn was awake - he would know what to do”
“I think we should wait for the storm to settle and then send the best
swimmer out to swim for the mainland and then walk for help”
“You are crazy! That water is freezing and its 120km walk to the
nearest habitation”
“Athletes swim the fjords all the time. 120 km is only 3 days walking. I’m for making an escape”
“I think we should make a raft and try and float our way to land.”
“You can both drown if you like. I’m staying put”
“Not me, I’m staying put, at least until the storm ends.”
15.34 HRS on Sunday 5th of June.
You are all still in the cove sheltering from the storm. The winds seem to
be dropping, but so does the temperature. What was a sunny warm day
now feels freezing and it has started pouring with rain. Bjørn does not
look good and you are all worried about him.
362
Appendicies
You are all dressed in lightweight summer clothes or bathing suits. One
of you has a pair of deck shoes on, the rest are barefoot. Collectively,
your possessions are: 12670 NOK in bills, 124 NOK in coins, 2
handkerchiefs, 1 scarf, one pair of nail scissors and one pack of cosmetics
with a mirror.
Given this situation, the resources that you have, the action alternatives
which you have and the salvaged items (all listed on pages 7 and 8) what
will you do?
You should assume the following:
1. You are the people in the situation – only the six described.
2. All salvaged items are in working order, but wet.
3. You will try to survive, keeping all six of you alive.
4. The captain of the boat has a broken arm and concussion.
Instructions
Now that you have survived the shipwreck, it is critical that you decide
how to deal with the situation.
First
Individually and without discussing The Situation, the ACTION
ALTERNATIVES, or the SALVAGEABLE ITEMS, with anyone, rank each
according to how important it is to your survival (Steps A to I).
Then
After everyone has finished the individual work, by group consensus,
rank each ACTION ALTERNATIVE and each SALVAGEABLE ITEM
according to how important your group thinks it is to your groups survival:
Achieving group consensus means arriving at sufficient agreement, not
necessarily unanimity (D to M).
363
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Action Alternatives
STEP A STEP B STEP C
Your
Expert Difference
Rank
Rank A to B
0
364
STEP G
Your Total
ACTIONALTERNATIVES
STEP D STEP E STEP F
Team Expert Difference
Rank
Rank
D to E
0
Develop techniques for signalling
0
0
Send representatives to nearest inhabited island
0
0
Do nothing until the storm has subsided
0
0
Search for food and water
0
0
Seek shelter in the cove
0
0
Give first aid to the injured
0
0
Look for vantage points for signalling
0
0
Send out two explorers to research the island
0
0
Decide on a leadership strategy and
start planning what needs to be done
0
0
0
STEP G
Team Total
0
Appendicies
Salvageable Items
STEP I
Your
Rank
0
STEP J STEP K
Expert Difference
Rank
I toJ
SALVAGEABLE ITEMS
STEP M STEP N STEP O
Team Expert Difference
Rank
Rank M to N
0
Two six packs of beer
0
0
Various pieces of stranded rigging wire 5’ to 20’ long
0
0
One scuba mask
0
0
Four bottles of SPF 5 suntan lotion
0
0
6’ x 14’ blue canvas canopy
0
0
One standard size plastic cooler
0
0
Yachtsman’s guide to the Fjords
0
0
Two orange life jackets
0
0
80’ nylon rope
0
0
Seven cans of food – Meat balls , baked beans,
peas, chicken chunks, fish soup and sardines
0
0
SwissArmy knife with multiple blades,
including a can opener
0
0
One pair of flippers
0
STEP L
Your Total
0 Four wrist watches, two of which are underwater types
0
0
One dinghy paddle
0
0
Three float seat cushions
0
0
0
STEP P
Team Total
0
365
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Expert Rankings
The Expert
Svein ‘IsBjørn’ Tanjun has 36 years of active military service including
many years as senior instructor at the elite Norwegian Survival, Evasion,
Resistance and Escape school. He has taught courses on survival in the
northern Scandinavian climate to the US Army Rangers, US Airborne,
US Navy SEALs, The Canadian Winter Warfare Unit, The British Royal
Marines and various other European NATO elite units.
The Expert’s Basic Guidelines
Stay Put & Keep Busy!
Most rescues have been made when the lost or stranded party stays in
the same spot and makes themselves easily found. Experience has shown
that this action has many advantages, enabling you to conserve food,
water and other supplies while maintaining all of your stores and focusing
solely on surviving and making yourself visible
Make a Fire
Making a fire for warmth and for signaling is of vital importance
Find or Create Shelter
The need for adequate shelter cannot be over emphasised. One cold
windy night in the open without fire or shelter could be your last. The
type of shelter that you construct will depend on the season and terrain
in which you find yourself. You should be adaptable and able to construct
several types of shelter depending on your circumstances
Water
About 80% of your body is water, and your vital life processes center
around the balance of intake and output of this most vital of elements.
Water is needed for normal functioning and when loss of water exceeds
intake, dehydration results. It is critical to maintain this balance in order
to avoid losing bodily efficiency.
366
Appendicies
Food
One of the most important things to know about food is that it is not as
important as you might think. Depending on your activity, you can go
without food for several weeks. Food can in fact work to your detriment
if you are low on your most critical resource, water. However eventually
you must eat and frequently, if you are stranded, this means you must
generate your own food sources
Create Some Signals
When you are stranded your chief objective should be to make yourself
as visible as possible to rescue parties. Once you have secured your
surroundings make sure to immediately develop your “signalling network”
Fire
Light by night
Smoke By day
Mirrors or Reflective Items
These are used to signal with when rescue is in sight.
Sound or Noise Production
These are used to attract attention in case vocal cords are dried and not
functioning. Carving a whistle can be a worthwhile distraction while you
wait to be rescued.
The Experts Action Rankings
1. Give First Aid to the Injured
2. Decide on a Leadership Strategy and Start Planning What Needs to
be Done
3. Send out Two explorers to research the Island
4. Seek Shelter in the Cove
5. Develop Techniques for Signalling
6. Look For Vantage points for signalling
7. Search for Food and Water
8. Do Nothing Until the Storm has Subsided
367
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
The Experts Salvageable Items Ranking
1. 6 x 14 blue canvas canopy
Shelter and water collection. Parts may be cut from the main canopy to
make solar water stills. This with the knife below is the most useful items on
this list. With these two items the chances of survival and rescue reach 86%.
2. Swiss Army knife with multiple tools
Best all round tool for survival. Shiny blade surface can act as a signal
device.
3. One Dinghy Paddle
Acts as a support for the canvas.
4. Seven cans of food
Can be eaten, but once opened can also act as water containers.
5. 80' of nylon rope
Can tie the canvas together - also acts as wicks for lights.
6. Two orange life jackets
When set on fire these will make a large smoke trail which can be seen
for many kilometres
7. Three flotation cushions
Likewise can be burnt to raise a smoke trail.
8. One Scuba Mask
The glass can be used as a light signal by catching the suns rays from the
glass. Might also be useful to smash and have additional knives. The
water is so deep and cold death by hypothermia would be inevitable for
anyone attempting to use the mask to swim.
368
Appendicies
9. Various Pieces of rigging wire
Might try to use these for traps, but frankly the chances of catching
anything large enough to reward the effort is unlikely.
10. One standard size plastic cooler
Good for storing water and fresh food - but since both will be very rare
you might use it as a basket to carry firewood from the trees and driftwood.
11. One pair of flippers
Could be used for digging or as smoke producing fuel. The water is so
deep and cold death by hypothermia would be inevitable for anyone
attempting to use the flippers to swim.
12. Four Bottles of Suntan Oil
You could use these as fuel to raise smoke from the fire or as fuel for
wick based lights using lengths of the nylon cord as a wick.
13. Two Six Packs of Beer
If you poured the beer away and used the cans to store water they might
be useful. The danger in drinking the beer is the dehydration caused by
the alcohol.
14. Four Wrist Watches
Might be useful for signalling using the watch glass.
15. Yachtman’s guide to the fjords
Might be used for fuel for starting the fire or toilet paper. The danger
would be if the book tempted people to try and escape. It is almost
certain they would perish in the icy waters of the fjord.
369
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix W:
Help Page Screen Shots
Figure 67 Help page for
the File Holder tool
Figure 68 Help page for
the ToDoList tool
370
Appendicies
Figure 69
Help page for the Page function
Figure 70
Help page for how to Save Rooms
371
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix Y:
Administration Interface in TeamWave Workplace
These are additional screen shots of the Administration Interface in
TeamWave Workplace
Figure 71
372
Administrator interface for administering conferences
Appendicies
Figure 72
Administrator interface for administering templates
Figure 73
Administrator interface for administering repositories
373
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix Y:
Usability Study Interview Guide
Goal of the interview
The goal of the interview is to get feedback on how efficient TeamWave
has been with respect to achievement of goals and what was needed/
required to reach these goal.
Various functions
First, I will present various functions that I think a groupware tool
should provide. Then, you will get to give your feedback with
respect to how you think Teamwave fulfilled these functions. The
functions are:
Communication: The tool should faciltate both synchronous and
asynchronous communication between the team members.
Administration: The tool should provide functions to support the team
in administration tasks such as scheduling meetings, distribution of work
tasks etc.
Production: The tool should support the team in production of artefacts such as text documents, images, web ages etc.
Presentation: The tool should facilitate the presentation and exchance
of various types of information between the team members.
Navigating: The tool should provide guidance for how to use the tool
and how to manouver within it.
374
Appendicies
Synchronous communication
• To what extent do you think TeamWave made it possible for the group
to communicate synchronously?
• What was the quality of this communication?
• Which tools did you use for synchronous communication?
• Do you have any suggestions for ways the synchronous communication
could have been improved?
Asynchronous communication
• Did TeamWave make it possible for the team to communicate
asynchronous?
• What was the quality of this communication?
• Which tools did you use for asynchronous communication?
• Do you have any suggestions for ways the asynchronous communication
could have been improved?
Administration
• Do you think that TeamWave supported the team in the administration
of the team work?
• If no:
• How did you administrate the teamwork?
• Otherwise:
• Which tools did you use to support administrative tasks?
• Are you satisfied in the way the team administrated the work, or did
you experience any problems?
Production
• Did you, through the support of TeamWave, manage to produce the
elements you wanted to include in the task (such as documents, images
etc.)?
• Which tools did you use to support these tasks?
• How well did these tools support production?
• Please report tools that you felt were missing, if any
Presentation
• To what extent did TeamWave support you in presenting information
for other team members?
• Which tools did you use as aids when presenting information?
• How well did these tools perform?
375
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Navigating
• Was it ok to navigate within the various TeamWave rooms?
• How did you manage in finding the other members of your team (easy
or difficult)?
Help
• To what extent did you use TeamWave’s help system?
• Did you use and of the web based help pages?
TeamWave in general
• Are there any other aspects of TeamWave that you want to comment
on? (not necessarily the specific tools, but rather some words on your
experiences in using TeamWave as a GroupWare tool.
Finally
• Did you make use of the team addresses when sending email to each
others, or did you also make use of private email addresses?
• Are you aware of any insecurities within the team with respect to what
was meant with ”collaboration” (such as everyone having to be logged
on TeamWave simultaneously to be able to take part in the
collaboration)?
376
Appendicies
Norsk
Formålet med intervjuet
Det jeg ønsker å se på i dette intervjuet er hvor effektivt dere synes
TeamWave har vært. Når det gjelder effektivitet vil jeg se på både effektivitet i forhold til måloppnåelse og effektivitet i forhold til hva som kreves for å nå dette målet (efficiency).
Ulike funksjoner
Jeg vil først ta for meg ulike funksjoner som jeg mener et gruppeverktøy
bør ha, og se på hvor godt du synes TeamWave fylte disse funksjonene.
Disse funksjonene er:
Kommunikasjon: At verktøyet gjør det mulig for gruppemedlemmene
å kommunisere både synkront og asynkront.
Administrasjon: At verktøyet støtter gruppen i administrering av gruppearbeidet, f.eks avtale møter, fordeling av oppgaver o.l.
Produksjon: At verktøyet gjør det mulig for gruppen å produsere noe,
f. eks tekstdokumenter, bilder, websider o.l.
Presentasjon: At verktøyet gjør det mulig for gruppemedlemmene å
presentere ulike typer informasjon for hverandre.
Navigering: At verktøyet støtter gruppemedlemmene i å finne frem
internt i verktøyet. (evt også eksternt?)
Synkron kommunikasjon
• I hvilken grad synes du at TeamWave gjorde det mulig for gruppen å
kommunisere synkront?
• Hvor godt synes du denne kommunikasjonen fungerte?
• Hvilke verktøy brukte dere for å kommunisere synkront?
• Har du noen forslag til hva som kunne vært gjort for at den synkrone
kommunikasjonen kunne fungert bedre?
377
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Asynkron kommunikasjon
• Synes du at TeamWave gjorde det mulig for gruppen å kommunisere
asynkront?
• Hvor godt synes du denne kommunikasjonen fungerte?
• Hvilke verktøy brukte dere for å kommunisere asynkront?
• Har du noen forslag til hva som kunne vært gjort for at den asynkrone
kommunikasjonen kunne fungert bedre?
Administrasjon
• Synes du TeamWave støttet gruppen i administreringen av gruppearbeidet?
• Hvis nei:
• Hvordan administrerte de gruppearbeidet?
• Ellers:
• Hvilke verktøy brukte dere til dette?
• Er du fornøyd med måten gruppen fikk administrert gruppearbeidet
eller fikk dere noen problemer?
Produksjon
• Klarte dere ved hjelp av TeamWave å produsere de elementene dere
ønsket å ha med i oppgaven? (f.eks. dokumenter, bilder o.l.)
• Hvilke verktøy brukte dere til dette?
• Hvor godt synes du disse verktøyene fungerte til dette?
• Var det noen verktøy/funksjoner du savnet?
Presentasjon
• I hvilken grad synes du at TeamWave gjorde det mulig for deg å
presentere informasjon for andre?
• Hvilke verktøy brukte dere til dette?
• Hvor godt synes du disse verktøyene fungerte?
Navigering
• Gikk det greit å navigere mellom de ulike rommene i TeamWave?
• Hvordan synes du det var å finne fram til de andre medlemmene i
gruppen din? (lett eller vanskelig)
Hjelp
• I hvilken grad brukte dere hjelpesystemet internt i TeamWave?
• Brukte dere de web-baserte hjelpesidene?
378
Appendicies
Generelt om TeamWave
• Er det noen andre aspekt ved TeamWave enn de vi har snakket om til
nå du ønsker å kommentere? (Da tenker jeg ikke nødvendigvis på de
spesifikke verktøyene, men mer generelt om hvordan du synes det
har vært å bruke TeamWave for å løse oppgaven deres.)
Avslutning
• Brukte dere team-adressen når gruppen sendte mail til hverandre,
eller brukte dere også private mail-adresser?
• Følte du at det var noe usikkerhet i gruppen om hva som menes med
collaboration? (f.eks. at alle måtte være logget på TW samtidig for å
samarbeide)
379
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix Z:
Interview Guide for How Students Organise their
Work (English)
Interview #:
Date:
Interviewee:
Interwiew Type:
General
• How has the assignment work and the group functioned?
* With respect to geographical distribution
* TW vs face-to-face meetings
* Difference from other collaboration forms - advantages/
disadvantages
Division of labour, roles, rules
• Who did what?
* Why?
* How?
• How was the division of labour decided?
* Discussion?
* Previous experience/education?
* Wishes?
* Agreement?
• Meets and collaboration
* Group vs individual
* Showing up at meetings - problems? (Especially in the first phase,
did this have implications for collaboration in later phases?)
* Why did you work so much on your own? (Easier, collaboration
problems?)
• Interdependencies vs flexability
380
Appendicies
Communication
• Communication forms
* Mail, TW - Others? Why?
* TW – advantanges, disadvantages, problems
* Server problems – How did the server work?
* Geographical distribution – What did this mean?
• General on TW as a communications tool
Integration
• How did the assignment work function in relation to your other
everyday tasks?
* Job, family, school, free time activities
• How did this effect the assignment work/group collaboration?
* Division of labour
* Meeting frequency/Meeting times
* Communication forms
* Comprehensiveness of the assignment (did you have to reduce the
comprehensiveness?)
Own Background
• Experience, knowledge, education
* Was your knowledge valuable, was it used?
* Were you able to contribute things that the others did not know
about?
* Did you/your group make use of knowledge the others had?
* Genuine interdependence?
Goal Setting
• What did you understand the goals of VisArt to be?
• What was your goal for participation in VisArt? Was it changed
underway? If yes, when, why?
• Engagement, what kept you engaged/not engaged in the work??
Evaluation of VisArt / Own experience
• What is your experience with VisArt?
Help Functions
• Did you use any help functions?
381
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Intervjuguiden (Norsk)
Intervju #:
Dato:
Informant:
Intervjutype:
Generelt
• Hvordan har arbeidet med oppgaven og gruppen fungert?
* Med hensyn til geografisk spredning
* TW vs face-to-face møter
* Forskjell fra andre samarbeidsformer - fordeler/ulemper
Arbeidsdeling, roller, regler
• Hvem har gjort hva?
* Hvorfor?
* Hvorda?
• Hvordan ble arbeidsdelingen fastsatt?
* Diskusjon?
* Tidligere erfaring/utdannelse?
* Ønsker?
* Enighet?
• Møter og samarbeid
* Gruppe vs enkeltperson
* Oppmøte - problemer? (Spesielt i oppgavens første del, hadde
dette betydning for samarbeidet i senere faser?)
* Hvorfor så mye arbeid på egenhånd? (Enklere, samarbeidsproblemer?)
• Interdependencies vs fleksibilitet
Kommunikasjon
• Kommunikasjonsformer
* Mail, TW - andre? Hvorfor?
* TW - fordeler, ulemper, problemer
* Serverproblemer - hvordan virket det inn?
* Geografisk spredning - hva betyr det?
• Generelt om TW som kommunikasjonsverktøy
382
Appendicies
Integrasjon
• Hvordan fungerte arbeidet med oppgaven i forhold til andre gjøremål i hverdagen?
* Jobb, familie, skole, fritidsaktiviteter
• Hvordan påvirket dette arbeidet med oppgaven/gruppearbeidet?
* Arbeidsdeling
* Møtefrekvens/-tidspunkter
* Kommunikasjonsformer
* Oppgavens omfang (Reduserte oppgavens omfang etterhvert)
Egen bakgrunn
• Erfaringer, kunnskap, utdanning
* Ble din kunnskap utnyttet, ble den brukt?
* Bidro du med ting de andre ikke kunne noe om?
* Dro du/gruppen nytte av kunnskap de andre satt inne med?
* Genuine interdependence?
Målsetting
• Hva oppfattet du som målsettingen med VisArt?
• Hva var din målsetting med å delta i VisArt? Ble den endret underveis? Hvis ja, når, hvorfor?
• Engasjement, hva gjorde at du engasjerte deg / ikke engasjerte deg
i arbeidet?
Vurdering av VisArt/egne erfaringer
• Hva er dine erfaringer med VisArt?
Hjelpefunksjoner
Brukte du noen hjelpefunksjoner under arbeidet?
383
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
Appendix AA:
Interview Guide for How Instructors and Facilitators
Organise their Work (English)
Interview #:
Date:
Interviewee:
1. General
How has working with the organisation of VisArt functioned?
- What has been going on, in general
- Who has been involved
- Planning and background
2. Division of labour:
Who has been doing what in the scenario? (Generally speaking)
- How did you arrive at the particular division of labour.
- What have been your tasks? (Specifically.)
- What has been your role in the scenario?
- Can you give examples of tasks that have been routinized in the
duration of the scenario?
3. Communication
What forms of communication have been used?
- What tools were used in communication?
- Were there any specific challenges in communicating with the District
Colleges?
- Did you use TeamWave to inform yourself about the work of the
students?
- Did you use TeamWave to communicate with the other instructors?
4. Rules, tacit/explicit:
Have there been any specific rules of interaction between the students?
- Tacit/ Explicit
384
Appendicies
5. Social context:
What other work has occupied you during the scenario?
- How has it influenced the work on VisArt?
- What was your background before taking part in the scenario?
6. Goal Setting
What was your motivation for taking part in the scenario?
- What was the goal for the VisArt scenario?
Intervjuguide for VisArt
Interview #:
(Norsk)
Dato:
Informant:
1. Generelt:
Hvordan har arbeidet med organiseringen av VisArt-scenariet fungert?
- Hva har foregått, generelt sett?
- Hvem har vært involvert?
- Planlegging og bakgrunn
2. Arbeidsdeling:
Hvem har gjort hva i scenariet? (generelt.)
- Hvordan ble man enige om fordelingen av arbeidet?
- Hva har vært din oppgave? (spesifikt.)
- Hvilken rolle har du hatt i scenariet?
- Kan du gi eksempel på typer av arbeidsoppgaver som har blitt rutinisert
i gjennomføringen av scenariet?
3. Kommunikasjon:
Hvilke kommunikasjonsformer ble benyttet?
- Hvilke verktøy ble benyttet i kommunikasjonen?
- Var det spesielle utfordringer knyttet til å kommunisere med høyskolene?
- Ble TeamWave brukt til å gi tilbakemelding om hva studentene foretok seg ?
- Ble TW brukt til å kommunisere instruktørene imellom?
385
Project DoCTA: Design and use of Collaborative Telelearning Artefacts
4 Regler, uformelle/formelle
Har det eksistert spesielle regler instruktørene imellom?
- Uttalte/uuttalte
5. Kontekst:
Hvilke andre gjøremål har informanten hatt i løpet av scenariet?
- Hvordan har det påvirket arbeidet med VisArt?
- Hvilken bakgrunn har informanten før deltakelse i scenariet?
6. Målsetning
Hva var din målsetning med å delta i VisArt scenariet?
- Hva var målsetningen med VisArt scenariet?
386