* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
Artificial Intelligence Review (1986) 1, 43-52 Intelligent tutoring systems survey M. Yazdani Department of Computer Science, University of Exeter, UK Abstract Simple computer assisted instruction (CAI) systems suffer from the fact that in general they do not know the subject matter they are teaching . Intelligent tutoring systems (ITS) use artificial intelligence (Al) formalisms to represent knowledge in order to improve on CAI systems . We survey a number of systems developed, and the emerging architecture for ITS development . A general summary Computer assisted instruction (CAI) is a mature technology . The use of `author languages' makes construction of such systems reasonably straightforward . The main problem with CAI systems is the shallow representation of knowledge of the domain in which they teach, and the fact that they are suited to teaching specific expertise and not abstract problem-solving activity . The first section of this report describes the difference between intelligent tutoring systems (ITS) and CAI . Most ITSs have been designed and remain as prototypes . This report presents a survey of the most well-known examples with details of the subject matter, the principle aim of their design and their outstanding features . The only two systems which are commercially available and in everyday use are in the domain of computer programming. The second section presents an exposition of these two systems : PROUST and the LISP tutor . Micro-PROUST is available as public domain software from BYTEnet listings and commercially from Park Row Software (1418 Park Row) of San Diego, CA 92037 . Neither the problem-description language used for PROUST nor the full PROUST system are available commercially . The LISP tutor is commercially available from Advanced Computer Tutoring Inc . (ACT) (701 Amberson Avenue) of Pittsburgh, PA 15232 at $20,000 . The GRAPES production system language, used as the ideal student model in the LISP tutor and other ACT tutors, is also available separately from Advanced Computer Tutoring Inc . at a price to be negotiated with each individual client . There does not seem to be an agreement on an architecture for ITS . It is proposed, however, that John Anderson's proposal, presented in Section 2, has to be considered as the most promising approach . Following on from the absence of an agreed architecture for ITS there is also a lack of software tools for building such systems . This means that the production of 43 44 M. Yazdani an ITS would be at least an order of magnitude more expensive than the production of a CAI system . 1. Computer assisted instruction (CAI) Computer assisted instruction has followed an evolutionary path since it was started in the 1950's with simple `linear programs' . In the 1960's it was felt that one could use the student's response to control the material that the student would be shown . In this way students should learn more effectively as they attempt problems of an appropriate difficulty, rather than wading through some systematic exploration . The `branching programs' therefore offered corrective feedback as well as adapting their teaching to students' responses . However, the task of designing teaching materials for such systems was impossibly large . This led to the birth of author languages' : specific languages suitable for the development of CAI material . In the 1970's a new level of sophistication was discovered in the design of CAI systems where, in some domains such as arithmetic, it became possible to generate the teaching material itself by computer. Such 'generative' systems could answer some of the questions asked by the students, and incorporate some measurement of difficulty of the task . By examining the development of algorithms for CAI over the last 30 years, we see that they have improved the richness of feedback and the degree of individualization offered to students . The main problem is the inherent impoverishment of their knowledge . In generative systems there is a mismatch between the program's internal processes (Boolean arithmetic) and those of the student's cognitive processes (rules and tables) . None of these systems has human-like knowledge of the domain it is teaching, nor can it answer serious questions from the students as to "why" and "how" the task is performed . Intelligent tutoring systems started as an enterprise which attempted to deal with the shortcomings of generative systems, and can be seen as the intelligent CAI of the 1980's . This enterprise has benefited from the work of researchers in the field of artificial intelligence (AI) who have had a long-standing preoccupation with the problem of how best to represent knowledge within an intelligent system . The notion that the tutoring program itself can solve the problem which it is setting for the student, and in a way similar to that of the student, is the basis of a large number of ITSs . Most ITSs (see Sleeman & Brown, 1982) have been designed and remain as prototypes . There are some notable exceptions among them : SOPHIE (Brown, Burton & de Kleer, 1982) is the oldest and most well known . SOPHIE was sponsored by the US Department of Defence (AFHRL, ARPA, Tri Services) . After limited use for on-site job training over the ARPA network for two years it is now no longer maintained . The three stages of development of SOPHIE (I-III) incorporate the most intensive attempt at building a complete ITS . The American Air Force interest was in the use of computers in advanced trouble shooting, particularly in a laboratory setting . The intention of the researchers in developing the system was to explore an interactive ITS Survey 45 learning environment : one which would encourage the explicit development of hypotheses by the student involved in problem solving, and which would in turn be communicated to the machine and subjected to critical analysis . SOPHIE uses a general-purpose electronic simular (Nagel & Pederson, 1973) in order to provide a simulation of the domain both for the student and itself . The key idea is to construct and 'run' an experiment in order to 'see' what happens, as opposed to the logical deduction of an answer . This is achieved by the use of an inbuilt articulate expert trouble shooter. It enables the student to insert arbitrary faults into a circuit and to watch the expert system locate them . There is a growing need for the training of personnel in the use of computers, and any form of automated support system for novice programmers can speed up this process . Further, the domain of computer programming, being a very structured one, makes it the most suitable test-bed for ITS development . It is not surprising that the only two ITSs which are currently offered for sale are both in this area, one for LISP (Anderson & Reiser, 1985) and the other for Pascal (Johnson & Soloway, 1985) . Finding syntactic errors in computer programs is a reasonably straightforward job . The reporting of these errors, in a form which will make it possible for the novice to learn more about the structure of the language, is not that straightforward . PROUST (Program Understanding for Students) is an intelligent tutoring system which finds errors ('bugs') in Pascal programs written by novice programmers . It is not confined to a narrow class of error, but is designed to find every bug in most beginners' programs . It is claimed (Johnson & Soloway, 1985) that PROUST can currently identify 70% of all the bugs in the programs that students write for moderately complex programming assignments . Once the bug has been identified, PROUST determines how the bug can be corrected, and suggests why the bug arose in the first place . PROUST, therefore, is part of a wider system which assigns exercises to students, analyses their work and gives them helpful suggestions . If an automatic tutor is to cope with variations and different types of student errors, it must understand what the programmer is trying to do . PROUST achieves its level of competence by being provided, in advance, with a description of the problems set for the students . Each of the problems, which is distributed to students, is also coded in a frame-based problem-description language and added to PROUST's library . PROUST also has a further knowledge base of common bugs in Pascal programs . Hence, as long as it knows what problem the student is trying to solve, and what possible mistakes are possible in the language, it can identify them in the students' various programs . PROUST synthesizes each program, searching for the corresponding problem description in the library, and making a hypothesis about the methods by which programmers may solve each part of the problem . If one of these hypotheses fits the student's code, then PROUST concludes that the student is correct . If not, it checks ibrary of common bugs to see if any of them fits the code . PROUST has been tested on a large number of Yale University undergraduates . It has also been used on a bank of recordings of student programs submitted to the Pascal compiler . PROUST has managed to score well over 70% in tests to identify all the bugs in a Pascal program . However, "17% of programs are analysed partially 46 M .Yazdani and 4% of the programs deviated from PROUST's expectations so drastically, it could not analyse them at all" (Johnson & Soloway, 1985) . A major problem with PROUST is that when it fails to understand a program completely, its ability to recognize bugs deteriorates dramatically . This indicates the sensitive role of a complete problem-description library . PROUST has been developed at a cost of half a million dollars over a 4-year period at Yale University with two programming assistants . It consists of 15,000 lines of LISP code (4 megabytes of memory) running on a DEC11VAX 750 . It takes 3-5 minutes to run . Micro-PROUST is a version of the system running on an IBM PC (512K) in Golden Common LISP . It is claimed to have taken one programmer 2 months to produce and only has one-fifth of the bug catalogue of PROUST, taking only 90 seconds to run but with an unknown rate of success . 2. ITS architecture As a result of the experimental nature of the work in the area of ITS, no clear general architecture for such systems can be identified as yet . However, the work of the Carnegie-Mellon University psychologist, John Anderson and his colleagues, on the LISP tutor (Anderson & Reiser, 1985) and the geometry tutor (Anderson, Boyle & Yost, 1985) strongly suggests a breakthrough . The underlying structure of Advanced Computer Tutoring principles (Anderson, Boyle & Reiser, 1985), used in systems for such diverse applications as LISP and Geometry, seems capable of supporting other subjects too . At the same time, Anderson's earlier work (1982) on Adaptive Control of Thought theory gives this approach a psychological plausibility . There are four components to ACT's ITS architecture : 1 Domain expert : this module is capable of actually solving problems in the domain . This is sometimes also referred to as the 'ideal student' model . 2 Bug catalogue : this is an extensive library of common misconceptions and errors in a domain . 3 Tutoring knowledge : this module contains the structure to teach the domain knowledge . 4 User interface : this is the module which administers interaction between the tutor and student . In contrast to the richness of student modelling of some other ITSs, the ACT tutors seem to incorporate a very dogmatic and authoritarian approach to education . The main driving force behind these tutoring systems is the detection of deviation from an ideal student model . Whenever the student makes a planning or coding error, the tutor guides the student back to the correct path . This obviously has some dangers, especially when the student is following a correct path but one which differs from the path that the system is following . Nevertheless, the LISP tutor seems to be able to turn problem solving episodes into learning experiences which they would not have been otherwise . The student's interface presents itself as a 'smart' screen editor . As long as the student does not make an error, the tutor remains quiet and is seen as no more than ITS Survey 47 an editor . If the student exhibits an error in his program, the system diagnoses the error and provides feedback in the form of a hint . The LISP tutor contains approximately 325 production rules concerned with the planning and coding of LISP programs, and 475 faulty versions of those rules . It is claimed to be "effective in diagnosing and responding to between 45% and 80% of students' errors" (Anderson & Reiser, 1985) . It can be run under VMS or UNIX operating systems of DEC VAXs . A single work station with 2 megabytes of memory could support one user, with 3-4 megabytes on the VAX 730 it can support two users and with 6-8 megabytes it could be used as a time-sharing program . The LISP tutor is commercially available from Advanced Computer Tutoring Inc . It is possible to argue that the architecture of ITS presented here is consistent with other proposals (Hartley & Sleeman, 1973 ; O'Shea et al ., 1984) . However, this is only on a superficial level as there seem to be major differences between the competing proposals on a number of issues ; most importantly, the role of student modelling. Hartley and Sleeman (1973) have suggested that ITS should normally have four distinct knowledge bases : 1 Knowledge of the task domain . 2 A model/history of the student's behaviour . 3 A list of possible teaching operations . 4 Mean-ends guidance rules which relate teaching decisions to conditions in the student model . This proposal differs from Anderson's (1985) inasmuch as it does not give the representation of misconceptions in the domain (the bug catalogue) primary importance, but instead introduces the student model as a primary component which is created for each individual user . Further, this proposal subsumes the user interface in a more tutoring oriented module, which includes guidance rules on how to carry out an interaction with the user . The five ring model presented by O'Shea and colleagues (1984), which bears some similarity to Hartley and Sleeman (1973), shows how the difference in the emphasis on student modelling and teaching strategies leads to an architecture which is radically different from Anderson's . This includes : 1 Student history. 2 Student model . 3 Teaching strategy . 4 Teaching generator. 5 Teaching administrator . resentation of the knowledge in the In this proposal the role of an ex domain (ideal model), or the common mi ptions in the domain catalogue), are underm ned in favour of emphas on the importance of various teaching skills . We believe that these three proposals can be viewed as points on a spectrum, where at one end Anderson's proposal is closer to the more open-ended exploratory learning environments of Papert (1980), and at the other end of the spectrum, O'Shea and colleagues' (1984) proposals are closer to the traditional CAI systems . 48 M . Yazdani The later ones sacrifice a rich representation of the knowledge domain in favour of emphasis on general purpose teaching skills . Learning environments Anderson's proposal Hartley & Sleeman's proposal O'Shea et al's proposal Traditional CAI The choice of a position on the above spectrum is not simply a matter of the convictions of the individual researchers, but is influenced by the nature of the expertise which is to be taught . Exceptionally abstract and general concepts, such as model building, use of analogy, etc ., can be better taught within an exploratory learning environment through the construction of an appropriate computer-based microworld (Lawler, 1984) . The teaching of skills which are basically problem solving in a specific domain can be best achieved via problem solving monitors such as Anderson's . As the tasks become more concrete and specific the proposals of Hartley and Sleeman, O'Shea and colleagues and those of traditional CAI become more appealing . There is, however, one major drawback to this diversity of methods for the design of ITS . While the development of traditional CAI systems is greatly facilitated by the use of author languages, construction of an ITS still seems to be a one-off process . O'Shea and colleagues have the most concrete proposal for a tool kit for ITS, due to its closeness to traditional CAI, while any form of a counterpart for author languages in ITS seems remote . It is clear, however, that ITS requires powerful knowledge representation formalisms . Production systems, used in Anderson's work, as seem to be as good as any other while offering a degree of psychological plausibility . Some attempts are currently being made to build tutoring prototype frameworks, where the system could easily be changed from application in one domain to another similar domain . Davies and co-workers (1985) present one such framework which, although designed to teach the Highway code in the first place, is now used for teaching flight safety regulations to air traffic controllers . 3. Survey In this section we present a survey of intelligent tutoring systems which, although not exhaustive, is intended to be a source of reference for further study . ACE Subject: Aim : Nuclear magnetic spectroscopy Monitor deductive reasoning Features : Problem solving monitor, accepts natural language input System : MODULAR ONE Reference : Sleeman, D . H . & Hendley, R. J . (1982) ACE : a system which analyses compplex explanations. In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . ITS Survey BUGGY & DEBUGGY Subject : Arithmetic Aim : Diagnose bugs from behaviour Features : Procedural representation of misconceptions (bugs), hypothesis generation, problem generation system : LISP System : LISP Reference : Brown, R. R . (1982) Diagnosing bugs in simple procedural skills . In Intelligent Tutoring Systems (eds, D . Sleeman & J . S. Brown) Academic Press, New York . BLOCKS Subject : Blocks game Aim : Diagnosis System : LISP Reference : Brown, J . S . & Brown, R . R . (1978) A paradigmatic example of an artificially intelligent instructional system . International Journal of Man-Machine Studies, 10, 232-339 . FGA Subject : French grammar Aim : Analyse free-form French sentences Features : Separation of dictionary, grammar, parser and error reporting, general shell idea, human controlled teaching strategy System : PROLOG Reference : Barchan, J ., Woodmansee, B . J . & Yazdani, M . (1986) A Prolog-based tool for French grammar analysis, Instructional Science, 5 . G UIDON Subject : Medical diagnosis Aim : Using MYCIN for tutoring Features : Overlay student model, case method, separation of domain knowledge from teaching expertise System : LISP Reference : Clancey, W . J . (1979) Tutoring rules for guiding a case method dialogue, International Journal of Man-Machine Studies, 11, 25-49 . GEOMETRY Tutor Subject : Aim : Features : Geometry Monitoring geometry proof problems Use of production rules to represent 'ideal student model' and 'bug catalogue' System : Franz LISP Reference : Anderson, J . R., Boyle, C . F . & Yost, G . (1985) The Geometry Tutor . Proceedings of IJCAI-85 . INTEGRATION Subject : Calculus Aim : To deal with student initiated examples of symbolic integration Features : Self-improvement System : LISP Reference : Kimbal, R. (1982) A self-improving tutor for symbolic integration . In Intelligent Tutoring Systems (eds D . Sleeman and J . S. Brown), Academic Press, New York . 49 50 M . Yazdani LISP Tutor LISP programming Teaching of introductory LISP programming Using deviation from ideal student model Franz LISP on VAX Reference : Anderson, J . R. & Reiser, B . (1985) The LISP Tutor . Byte, 10(4), 159-175 . LMS (Pixie) Subject : Aim : Features : System : Subject : Aim : Features : Algebra equation solving Building student models Given problems and students' answers it hypothesises models for them ; uses rules and mal-rules . LISP System : Reference : Sleeman, D . A . (1983) Inferring student models for intelligent computeraided instruction . In Machine Learning (eds R . Michalsky, J . Carbonnel & T . Mitchell), Springer-Verlag/Toga Press, Stuttgart. MENO Subject : Pascal programming Aim : Tutoring novice programmers in the use of planning Features : Hierarchical representation of correct and incorrect plans System : LISP Reference : Woolf, B . & McDonald, D . D . (1984) Building a computer tutor : design issues . IEEE Computers, September issue, 61-73 . MACSYMA ADVISOR Subject : Aim : Use of MACSYMA Articulate users' misconceptions about MACSYMA Features : Representation of plans System : LISP Reference : Genesreth, M . R. (1977) An automated consultant for MACSYMA . Proceedings of IJCAI-77 . NEOMYCIN Subject: Aim : Features : Medical diagnosis Using expert systems for tutoring Separation of domain knowledge from teaching expertise, automatic explanation of experts' reasoning System : LISP Reference : Hasling, D . W ., Clancey, W . J . & Rennels, G. (1984) Strategic explanations for a diagnostic consultation system . PROUST Subject: Aim : Features : Pascal programming Automatic debugger and tutor Use of problem descriptions System : GCL LISP on IBM PC (micro-PROUST), LISP on VAXs Reference : Johnson, W . L. & Soloway, E . (1985) PROUST. Byte, 10(4), 179-190 . QUADRATIC Tutor Subject: Aim: Features : System : Calculus Teaching quadratic equations Teaching strategy represented as a set of production rules LISP Reference : O'Shea, T . (1982) A self-improving quadratic tutor . In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . ITS Survey 51 SCHOLAR Subject : Geography Aim : Provide mixed-initiative dialogue Features : Semantic network representation of knowledge System : LISP Reference : Carbonnel, J . R . & Collins, A . (1973) Natural semantics in artificial intelligence . Proceedings of IJCAI-73 and Proceedings of IJCAI-85 . SOPHIE Subject : Aim : Features : Electronic trouble shooting Teaching how an expert trouble shooter copes with rare faults Semantic grammar for natural language dialogue, qualitative knowledge plus simulation, multiple knowledge sources System : LISP Reference : Brown, J . S ., Burton, R. R . & deKleer, J . (1982) Pedagogical, natural language and knowledge engineering techniques in SOPHIE I, II and III . In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . SPADE Subject : Aim : LOGO programming To facilitate the acquisition of programming skills Intelligent editor which prompts the student with a menu of design alternatives Reference : Miller, M . L . (1982) A structured planning and debugging environment inferring student models for intelligent computer-aided instruction . In Intelligent Tutoring Systems (eds D . Sleeman & J . S. Brown), Academic Press, New York. STEAMER Features : Subject : Steam plant operation Aim : Convey qualitative model of a steam plant operation Features : Good graphics and mathematical model of the plant System : LISP Reference : Holland, J . D ., Hutchins, E . L . & Weitzmann, L . (1984) STEAMER : an interactive inspectable simulation-based training system, Al Magazine, 5(2) . TUTOR Subject : Aim : Features : Highway code Prototype framework for a wide variety of subjects Semantic grammar implemented in definite clause grammar, representing value clusters, 'what if' facility System : Prolog on VAX and IBM PC AT Reference : Davies, N ., Dickens, S . & Ford, L . (1985) TUTOR : a prototype ICAI system . In Research and Development in Expert Systems (ed . M . Bramer), Cambridge University Press, Cambridge . WEST Subject : How the West was won Aim : Drill and practice in arithmetic Features : Hierarchical representation of correct and incorrect plans System : PLATO Reference : Comparison of students' moves with experts' moves, student model and diagnostic strategies, tutoring expert . 52 M. Yazdani WHY Subject : Meteorology Aim : Tutoring students about processes involved in rainfall Features : Multiple representations in direct tuition System : LISP Reference : Stevens, A . & Goldin, S . F . (1982) Misconceptions in student understanding . In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . WUSOR Subject: Maze exploration game (Wumpus) Aim: Teaching logic and probability Features : Graph structure whose nodes represent rules System : LISP Reference : Goldstein, I . (1982) The genetic graph : a representation for evolution of procedural knowledge . In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . References Anderson, J . R . (1985) Skill Acquisition : compilation of weak method problem solutions . CarnegieMellon University . Anderson, J . R ., Boyle, C . F . & Reiser, B . J . (1985) Intelligent tutoring systems . Science, 228, 456-462 . Anderson, J . R ., Boyle, C . F . & Yost, G . (1985) The Geometry Tutor . Proceedings of IJCAI-85, . Anderson, J . R . & Reiser, B . J . (1985) The LISP Tutor. Byte, 10(4) . Brown, J . S ., Burton, R . R. & de Kleer, J . (1982) Pedagogical, natural language and knowledge engineering techniques in SOPHIE I, II and III . In Intelligent Tutoring Systems (eds D . Sleeman & J . S . Brown), Academic Press, New York . Davies, N . G ., Dickens, S . L. & Ford, L . (1985) TUTOR - a prototype ICAI system . In Research and Development in Expert Systems (ed . M . Bramer), Cambridge University Press, Cambridge . Hartley, J . R . & Sleeman, D. H . (1973) Towards intelligent teaching systems . International Journal of Man-Machine Studies, Johnson, W. L . & Soloway, E . (1985) PROUST. Byte, 10(4) . Lawler, R. W. (1984) Designing computerbased microworlds . In New Horizons in Edu Computing. (ed . M . Yazdani), Ellis Harwood, Chichester . Nagel, L. W . & Pedersen, D . O . (1973) Simulation program with integrated circuit emphasis . Proceedings of the 6th Midwest Symposium Circuit Theory, Waterloo, Canada. O'Shea, T ., Bornat, R ., du Boulay, B ., Eisenstad, M . & Page, I . (1984) Tools for creating intelligent computer tutors . In Human and Artificial Intelligence (eds . Elithor & Banerjii), North Holland . Papert, S . (1980) Mindstorms, Children, Computers and Powerful Ideas . Harvester Press/Basic Books, Brighton . Sleeman, D . & Brown, J . S . eds . (1982) Intelligent Tutoring Systems . Academic Press, New York .