Download Intelligent tutoring systems survey | SpringerLink

Artificial Intelligence
Review (1986) 1, 43-52
Intelligent tutoring systems survey
M. Yazdani
Department of Computer Science, University of Exeter, UK
Abstract Simple computer assisted instruction (CAI) systems suffer from
the fact that in general they do not know the subject matter they are
teaching . Intelligent tutoring systems (ITS) use artificial intelligence (Al)
formalisms to represent knowledge in order to improve on CAI systems .
We survey a number of systems developed, and the emerging architecture
for ITS development .
A general summary
Computer assisted instruction (CAI) is a mature technology . The use of `author
languages' makes construction of such systems reasonably straightforward . The
main problem with CAI systems is the shallow representation of knowledge of the
domain in which they teach, and the fact that they are suited to teaching specific
expertise and not abstract problem-solving activity . The first section of this report
describes the difference between intelligent tutoring systems (ITS) and CAI .
Most ITSs have been designed and remain as prototypes . This report presents a
survey of the most well-known examples with details of the subject matter, the
principle aim of their design and their outstanding features . The only two systems
which are commercially available and in everyday use are in the domain of
computer programming. The second section presents an exposition of these two
systems : PROUST and the LISP tutor .
Micro-PROUST is available as public domain software from BYTEnet listings and
commercially from Park Row Software (1418 Park Row) of San Diego, CA 92037 .
Neither the problem-description language used for PROUST nor the full PROUST
system are available commercially . The LISP tutor is commercially available from
Advanced Computer Tutoring Inc . (ACT) (701 Amberson Avenue) of Pittsburgh,
PA 15232 at $20,000 . The GRAPES production system language, used as the ideal
student model in the LISP tutor and other ACT tutors, is also available separately
from Advanced Computer Tutoring Inc . at a price to be negotiated with each
individual client .
There does not seem to be an agreement on an architecture for ITS . It is proposed,
however, that John Anderson's proposal, presented in Section 2, has to be considered as the most promising approach .
Following on from the absence of an agreed architecture for ITS there is also a
lack of software tools for building such systems . This means that the production of
43
44
M. Yazdani
an ITS would be at least an order of magnitude more expensive than the production
of a CAI system .
1.
Computer assisted instruction (CAI)
Computer assisted instruction has followed an evolutionary path since it was
started in the 1950's with simple `linear programs' . In the 1960's it was felt that one
could use the student's response to control the material that the student would be
shown . In this way students should learn more effectively as they attempt problems
of an appropriate difficulty, rather than wading through some systematic exploration . The `branching programs' therefore offered corrective feedback as well as
adapting their teaching to students' responses . However, the task of designing
teaching materials for such systems was impossibly large . This led to the birth of
author languages' : specific languages suitable for the development of CAI material .
In the 1970's a new level of sophistication was discovered in the design of CAI
systems where, in some domains such as arithmetic, it became possible to generate
the teaching material itself by computer. Such 'generative' systems could answer
some of the questions asked by the students, and incorporate some measurement of
difficulty of the task .
By examining the development of algorithms for CAI over the last 30 years, we
see that they have improved the richness of feedback and the degree of individualization offered to students . The main problem is the inherent impoverishment of
their knowledge . In generative systems there is a mismatch between the program's
internal processes (Boolean arithmetic) and those of the student's cognitive processes (rules and tables) . None of these systems has human-like knowledge of the
domain it is teaching, nor can it answer serious questions from the students as to
"why" and "how" the task is performed .
Intelligent tutoring systems started as an enterprise which attempted to deal with
the shortcomings of generative systems, and can be seen as the intelligent CAI of the
1980's . This enterprise has benefited from the work of researchers in the field of
artificial intelligence (AI) who have had a long-standing preoccupation with the
problem of how best to represent knowledge within an intelligent system .
The notion that the tutoring program itself can solve the problem which it is
setting for the student, and in a way similar to that of the student, is the basis of a
large number of ITSs .
Most ITSs (see Sleeman & Brown, 1982) have been designed and remain as
prototypes . There are some notable exceptions among them : SOPHIE (Brown,
Burton & de Kleer, 1982) is the oldest and most well known . SOPHIE was sponsored by the US Department of Defence (AFHRL, ARPA, Tri Services) . After limited
use for on-site job training over the ARPA network for two years it is now no longer
maintained .
The three stages of development of SOPHIE (I-III) incorporate the most intensive
attempt at building a complete ITS . The American Air Force interest was in the use
of computers in advanced trouble shooting, particularly in a laboratory setting . The
intention of the researchers in developing the system was to explore an interactive
ITS Survey
45
learning environment : one which would encourage the explicit development of
hypotheses by the student involved in problem solving, and which would in turn
be communicated to the machine and subjected to critical analysis .
SOPHIE uses a general-purpose electronic simular (Nagel & Pederson, 1973) in
order to provide a simulation of the domain both for the student and itself . The key
idea is to construct and 'run' an experiment in order to 'see' what happens, as
opposed to the logical deduction of an answer . This is achieved by the use of an
inbuilt articulate expert trouble shooter. It enables the student to insert arbitrary
faults into a circuit and to watch the expert system locate them .
There is a growing need for the training of personnel in the use of computers, and
any form of automated support system for novice programmers can speed up this
process . Further, the domain of computer programming, being a very structured
one, makes it the most suitable test-bed for ITS development . It is not surprising
that the only two ITSs which are currently offered for sale are both in this area, one
for LISP (Anderson & Reiser, 1985) and the other for Pascal (Johnson & Soloway,
1985) .
Finding syntactic errors in computer programs is a reasonably straightforward
job . The reporting of these errors, in a form which will make it possible for the
novice to learn more about the structure of the language, is not that straightforward .
PROUST (Program Understanding for Students) is an intelligent tutoring system
which finds errors ('bugs') in Pascal programs written by novice programmers . It is
not confined to a narrow class of error, but is designed to find every bug in most
beginners' programs . It is claimed (Johnson & Soloway, 1985) that PROUST can
currently identify 70% of all the bugs in the programs that students write for
moderately complex programming assignments . Once the bug has been identified,
PROUST determines how the bug can be corrected, and suggests why the bug arose
in the first place . PROUST, therefore, is part of a wider system which assigns
exercises to students, analyses their work and gives them helpful suggestions .
If an automatic tutor is to cope with variations and different types of student
errors, it must understand what the programmer is trying to do . PROUST achieves
its level of competence by being provided, in advance, with a description of the
problems set for the students . Each of the problems, which is distributed to
students, is also coded in a frame-based problem-description language and added
to PROUST's library . PROUST also has a further knowledge base of common bugs
in Pascal programs . Hence, as long as it knows what problem the student is trying to
solve, and what possible mistakes are possible in the language, it can identify them
in the students' various programs .
PROUST synthesizes each program, searching for the corresponding problem
description in the library, and making a hypothesis about the methods by which
programmers may solve each part of the problem . If one of these hypotheses fits the
student's code, then PROUST concludes that the student is correct . If not, it checks
ibrary of common bugs to see if any of them fits the code .
PROUST has been tested on a large number of Yale University undergraduates . It
has also been used on a bank of recordings of student programs submitted to the
Pascal compiler . PROUST has managed to score well over 70% in tests to identify
all the bugs in a Pascal program . However, "17% of programs are analysed partially
46 M .Yazdani
and 4% of the programs deviated from PROUST's expectations so drastically, it
could not analyse them at all" (Johnson & Soloway, 1985) . A major problem with
PROUST is that when it fails to understand a program completely, its ability to
recognize bugs deteriorates dramatically . This indicates the sensitive role of a
complete problem-description library .
PROUST has been developed at a cost of half a million dollars over a 4-year
period at Yale University with two programming assistants . It consists of 15,000
lines of LISP code (4 megabytes of memory) running on a DEC11VAX 750 . It takes
3-5 minutes to run . Micro-PROUST is a version of the system running on an IBM
PC (512K) in Golden Common LISP . It is claimed to have taken one programmer 2
months to produce and only has one-fifth of the bug catalogue of PROUST, taking
only 90 seconds to run but with an unknown rate of success .
2.
ITS architecture
As a result of the experimental nature of the work in the area of ITS, no clear general
architecture for such systems can be identified as yet . However, the work of the
Carnegie-Mellon University psychologist, John Anderson and his colleagues, on
the LISP tutor (Anderson & Reiser, 1985) and the geometry tutor (Anderson, Boyle &
Yost, 1985) strongly suggests a breakthrough . The underlying structure of
Advanced Computer Tutoring principles (Anderson, Boyle & Reiser, 1985), used in
systems for such diverse applications as LISP and Geometry, seems capable of
supporting other subjects too . At the same time, Anderson's earlier work (1982)
on Adaptive Control of Thought theory gives this approach a psychological plausibility .
There are four components to ACT's ITS architecture :
1 Domain expert : this module is capable of actually solving problems in the
domain . This is sometimes also referred to as the 'ideal student' model .
2 Bug catalogue : this is an extensive library of common misconceptions and errors
in a domain .
3 Tutoring knowledge : this module contains the structure to teach the domain
knowledge .
4 User interface : this is the module which administers interaction between the
tutor and student .
In contrast to the richness of student modelling of some other ITSs, the ACT
tutors seem to incorporate a very dogmatic and authoritarian approach to education . The main driving force behind these tutoring systems is the detection of
deviation from an ideal student model . Whenever the student makes a planning or
coding error, the tutor guides the student back to the correct path . This obviously
has some dangers, especially when the student is following a correct path but one
which differs from the path that the system is following . Nevertheless, the LISP
tutor seems to be able to turn problem solving episodes into learning experiences
which they would not have been otherwise .
The student's interface presents itself as a 'smart' screen editor . As long as the
student does not make an error, the tutor remains quiet and is seen as no more than
ITS Survey 47
an editor . If the student exhibits an error in his program, the system diagnoses the
error and provides feedback in the form of a hint .
The LISP tutor contains approximately 325 production rules concerned with the
planning and coding of LISP programs, and 475 faulty versions of those rules . It is
claimed to be "effective in diagnosing and responding to between 45% and 80% of
students' errors" (Anderson & Reiser, 1985) . It can be run under VMS or UNIX
operating systems of DEC VAXs . A single work station with 2 megabytes of memory
could support one user, with 3-4 megabytes on the VAX 730 it can support two
users and with 6-8 megabytes it could be used as a time-sharing program . The LISP
tutor is commercially available from Advanced Computer Tutoring Inc .
It is possible to argue that the architecture of ITS presented here is consistent
with other proposals (Hartley & Sleeman, 1973 ; O'Shea et al ., 1984) . However, this
is only on a superficial level as there seem to be major differences between the
competing proposals on a number of issues ; most importantly, the role of student
modelling. Hartley and Sleeman (1973) have suggested that ITS should normally
have four distinct knowledge bases :
1 Knowledge of the task domain .
2 A model/history of the student's behaviour .
3 A list of possible teaching operations .
4 Mean-ends guidance rules which relate teaching decisions to conditions in the
student model .
This proposal differs from Anderson's (1985) inasmuch as it does not give the
representation of misconceptions in the domain (the bug catalogue) primary importance, but instead introduces the student model as a primary component which is
created for each individual user . Further, this proposal subsumes the user interface
in a more tutoring oriented module, which includes guidance rules on how to carry
out an interaction with the user .
The five ring model presented by O'Shea and colleagues (1984), which bears
some similarity to Hartley and Sleeman (1973), shows how the difference in the
emphasis on student modelling and teaching strategies leads to an architecture
which is radically different from Anderson's . This includes :
1 Student history.
2 Student model .
3 Teaching strategy .
4 Teaching generator.
5 Teaching administrator .
resentation of the knowledge in the
In this proposal the role of an ex
domain (ideal model), or the common mi
ptions in the domain
catalogue), are underm ned in favour of emphas on the importance of various
teaching skills .
We believe that these three proposals can be viewed as points on a spectrum,
where at one end Anderson's proposal is closer to the more open-ended exploratory
learning environments of Papert (1980), and at the other end of the spectrum,
O'Shea and colleagues' (1984) proposals are closer to the traditional CAI systems .
48 M . Yazdani
The later ones sacrifice a rich representation of the knowledge domain in favour of
emphasis on general purpose teaching skills .
Learning
environments
Anderson's
proposal
Hartley &
Sleeman's
proposal
O'Shea et al's
proposal
Traditional CAI
The choice of a position on the above spectrum is not simply a matter of the
convictions of the individual researchers, but is influenced by the nature of the
expertise which is to be taught . Exceptionally abstract and general concepts,
such as model building, use of analogy, etc ., can be better taught within an
exploratory learning environment through the construction of an appropriate
computer-based microworld (Lawler, 1984) . The teaching of skills which are
basically problem solving in a specific domain can be best achieved via problem solving monitors such as Anderson's . As the tasks become more concrete
and specific the proposals of Hartley and Sleeman, O'Shea and colleagues and
those of traditional CAI become more appealing .
There is, however, one major drawback to this diversity of methods for the
design of ITS . While the development of traditional CAI systems is greatly
facilitated by the use of author languages, construction of an ITS still seems to
be a one-off process . O'Shea and colleagues have the most concrete proposal for
a tool kit for ITS, due to its closeness to traditional CAI, while any form of a
counterpart for author languages in ITS seems remote . It is clear, however, that
ITS requires powerful knowledge representation formalisms . Production systems, used in Anderson's work, as seem to be as good as any other while
offering a degree of psychological plausibility .
Some attempts are currently being made to build tutoring prototype frameworks, where the system could easily be changed from application in one
domain to another similar domain . Davies and co-workers (1985) present one
such framework which, although designed to teach the Highway code in the
first place, is now used for teaching flight safety regulations to air traffic
controllers .
3. Survey
In this section we present a survey of intelligent tutoring systems which,
although not exhaustive, is intended to be a source of reference for further
study .
ACE
Subject:
Aim :
Nuclear magnetic spectroscopy
Monitor deductive reasoning
Features : Problem solving monitor, accepts natural language input
System :
MODULAR ONE
Reference : Sleeman, D . H . & Hendley, R. J . (1982) ACE : a system which analyses compplex explanations. In Intelligent Tutoring Systems (eds D . Sleeman & J . S .
Brown), Academic Press, New York .
ITS Survey
BUGGY & DEBUGGY
Subject :
Arithmetic
Aim :
Diagnose bugs from behaviour
Features : Procedural representation of misconceptions (bugs), hypothesis generation,
problem generation system : LISP
System :
LISP
Reference : Brown, R. R . (1982) Diagnosing bugs in simple procedural skills . In Intelligent Tutoring Systems (eds, D . Sleeman & J . S. Brown) Academic Press,
New York .
BLOCKS
Subject :
Blocks game
Aim :
Diagnosis
System :
LISP
Reference : Brown, J . S . & Brown, R . R . (1978) A paradigmatic example of an artificially
intelligent instructional system . International Journal of Man-Machine
Studies, 10, 232-339 .
FGA
Subject : French grammar
Aim :
Analyse free-form French sentences
Features : Separation of dictionary, grammar, parser and error reporting, general shell
idea, human controlled teaching strategy
System : PROLOG
Reference : Barchan, J ., Woodmansee, B . J . & Yazdani, M . (1986) A Prolog-based tool for
French grammar analysis, Instructional Science, 5 .
G UIDON
Subject :
Medical diagnosis
Aim :
Using MYCIN for tutoring
Features : Overlay student model, case method, separation of domain knowledge from
teaching expertise
System :
LISP
Reference : Clancey, W . J . (1979) Tutoring rules for guiding a case method dialogue,
International Journal of Man-Machine Studies, 11, 25-49 .
GEOMETRY Tutor
Subject :
Aim :
Features :
Geometry
Monitoring geometry proof problems
Use of production rules to represent 'ideal student model' and 'bug catalogue'
System : Franz LISP
Reference : Anderson, J . R., Boyle, C . F . & Yost, G . (1985) The Geometry Tutor . Proceedings of IJCAI-85 .
INTEGRATION
Subject :
Calculus
Aim :
To deal with student initiated examples of symbolic integration
Features : Self-improvement
System :
LISP
Reference : Kimbal, R. (1982) A self-improving tutor for symbolic integration . In Intelligent Tutoring Systems (eds D . Sleeman and J . S. Brown), Academic Press,
New York .
49
50
M . Yazdani
LISP Tutor
LISP programming
Teaching of introductory LISP programming
Using deviation from ideal student model
Franz LISP on VAX
Reference : Anderson, J . R. & Reiser, B . (1985) The LISP Tutor . Byte, 10(4),
159-175 .
LMS (Pixie)
Subject :
Aim :
Features :
System :
Subject :
Aim :
Features :
Algebra equation solving
Building student models
Given problems and students' answers it hypothesises models for them ;
uses rules and mal-rules .
LISP
System :
Reference : Sleeman, D . A . (1983) Inferring student models for intelligent computeraided instruction . In Machine Learning (eds R . Michalsky, J . Carbonnel & T .
Mitchell), Springer-Verlag/Toga Press, Stuttgart.
MENO
Subject :
Pascal programming
Aim :
Tutoring novice programmers in the use of planning
Features : Hierarchical representation of correct and incorrect plans
System :
LISP
Reference : Woolf, B . & McDonald, D . D . (1984) Building a computer tutor : design
issues . IEEE Computers, September issue, 61-73 .
MACSYMA ADVISOR
Subject :
Aim :
Use of MACSYMA
Articulate users' misconceptions about MACSYMA
Features : Representation of plans
System :
LISP
Reference : Genesreth, M . R. (1977) An automated consultant for MACSYMA .
Proceedings of IJCAI-77 .
NEOMYCIN
Subject:
Aim :
Features :
Medical diagnosis
Using expert systems for tutoring
Separation of domain knowledge from teaching expertise, automatic
explanation of experts' reasoning
System :
LISP
Reference : Hasling, D . W ., Clancey, W . J . & Rennels, G. (1984) Strategic explanations
for a diagnostic consultation system .
PROUST
Subject:
Aim :
Features :
Pascal programming
Automatic debugger and tutor
Use of problem descriptions
System :
GCL LISP on IBM PC (micro-PROUST), LISP on VAXs
Reference : Johnson, W . L. & Soloway, E . (1985) PROUST. Byte, 10(4), 179-190 .
QUADRATIC Tutor
Subject:
Aim:
Features :
System :
Calculus
Teaching quadratic equations
Teaching strategy represented as a set of production rules
LISP
Reference : O'Shea, T . (1982) A self-improving quadratic tutor . In Intelligent Tutoring
Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York .
ITS Survey
51
SCHOLAR
Subject :
Geography
Aim :
Provide mixed-initiative dialogue
Features : Semantic network representation of knowledge
System :
LISP
Reference : Carbonnel, J . R . & Collins, A . (1973) Natural semantics in artificial intelligence . Proceedings of IJCAI-73 and Proceedings of IJCAI-85 .
SOPHIE
Subject :
Aim :
Features :
Electronic trouble shooting
Teaching how an expert trouble shooter copes with rare faults
Semantic grammar for natural language dialogue, qualitative knowledge
plus simulation, multiple knowledge sources
System :
LISP
Reference : Brown, J . S ., Burton, R. R . & deKleer, J . (1982) Pedagogical, natural language
and knowledge engineering techniques in SOPHIE I, II and III . In Intelligent
Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New
York .
SPADE
Subject :
Aim :
LOGO programming
To facilitate the acquisition of programming skills
Intelligent editor which prompts the student with a menu of design
alternatives
Reference : Miller, M . L . (1982) A structured planning and debugging environment
inferring student models for intelligent computer-aided instruction . In
Intelligent Tutoring Systems (eds D . Sleeman & J . S. Brown), Academic
Press, New York.
STEAMER
Features :
Subject :
Steam plant operation
Aim :
Convey qualitative model of a steam plant operation
Features : Good graphics and mathematical model of the plant
System :
LISP
Reference : Holland, J . D ., Hutchins, E . L . & Weitzmann, L . (1984) STEAMER : an
interactive inspectable simulation-based training system, Al Magazine, 5(2) .
TUTOR
Subject :
Aim :
Features :
Highway code
Prototype framework for a wide variety of subjects
Semantic grammar implemented in definite clause grammar, representing
value clusters, 'what if' facility
System :
Prolog on VAX and IBM PC AT
Reference : Davies, N ., Dickens, S . & Ford, L . (1985) TUTOR : a prototype ICAI system .
In Research and Development in Expert Systems (ed . M . Bramer),
Cambridge University Press, Cambridge .
WEST
Subject :
How the West was won
Aim :
Drill and practice in arithmetic
Features : Hierarchical representation of correct and incorrect plans
System :
PLATO
Reference : Comparison of students' moves with experts' moves, student model and
diagnostic strategies, tutoring expert .
52
M. Yazdani
WHY
Subject : Meteorology
Aim :
Tutoring students about processes involved in rainfall
Features : Multiple representations in direct tuition
System :
LISP
Reference : Stevens, A . & Goldin, S . F . (1982) Misconceptions in student understanding .
In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic
Press, New York .
WUSOR
Subject: Maze exploration game (Wumpus)
Aim:
Teaching logic and probability
Features : Graph structure whose nodes represent rules
System :
LISP
Reference : Goldstein, I . (1982) The genetic graph : a representation for evolution of
procedural knowledge . In Intelligent Tutoring Systems (eds D . Sleeman &
J . S . Brown), Academic Press, New York .
References
Anderson, J . R . (1985) Skill Acquisition : compilation of weak method problem solutions . CarnegieMellon University .
Anderson, J . R ., Boyle, C . F . & Reiser, B . J . (1985) Intelligent tutoring systems . Science, 228,
456-462 .
Anderson, J . R ., Boyle, C . F . & Yost, G . (1985) The Geometry Tutor . Proceedings of IJCAI-85, .
Anderson, J . R . & Reiser, B . J . (1985) The LISP Tutor. Byte, 10(4) .
Brown, J . S ., Burton, R . R. & de Kleer, J . (1982) Pedagogical, natural language and knowledge
engineering techniques in SOPHIE I, II and III . In Intelligent Tutoring Systems (eds D . Sleeman
& J . S . Brown), Academic Press, New York .
Davies, N . G ., Dickens, S . L. & Ford, L . (1985) TUTOR - a prototype ICAI system . In Research and
Development in Expert Systems (ed . M . Bramer), Cambridge University Press, Cambridge .
Hartley, J . R . & Sleeman, D. H . (1973) Towards intelligent teaching systems . International Journal
of Man-Machine Studies,
Johnson, W. L . & Soloway, E . (1985) PROUST. Byte, 10(4) .
Lawler, R. W. (1984) Designing computerbased microworlds . In New Horizons in Edu
Computing. (ed . M . Yazdani), Ellis Harwood, Chichester .
Nagel, L. W . & Pedersen, D . O . (1973) Simulation program with integrated circuit emphasis .
Proceedings of the 6th Midwest Symposium Circuit Theory, Waterloo, Canada.
O'Shea, T ., Bornat, R ., du Boulay, B ., Eisenstad, M . & Page, I . (1984) Tools for creating intelligent
computer tutors . In Human and Artificial Intelligence (eds . Elithor & Banerjii), North Holland .
Papert, S . (1980) Mindstorms, Children, Computers and Powerful Ideas . Harvester Press/Basic
Books, Brighton .
Sleeman, D . & Brown, J . S . eds . (1982) Intelligent Tutoring Systems . Academic Press, New York .