Download TAILS: Enhanced Learning Through Experiencing Artificial

TAILS: Enhanced Learning Through Experiencing
Artificial Intelligence as a Lab Science
Stephanie E. August
Department of Electrical Engineering and Computer Science
Loyola Marymount University Los Angeles
[email protected]
Abstract
The TAILS project addresses three needs. First, literature on computer science
education reveals that artificial intelligence (AI) faculty are challenged to balance the
benefits of teaching undergraduates the intricacies of specific complex algorithms
against the benefits of exposing them to a wide variety of algorithms and applications.
Second, computer science needs to attract and retain students who have an
interpersonal orientation in general and women in particular. Third, to be fully literate,
students must be able to view software systems at many levels of abstraction.
TAILS is designed to develop a paradigm for teaching AI concepts by implementing an
experiment-based approach modeled after the lab sciences. AI and software
engineering course material are interwoven in an experiment paradigm to provide
experiential learning opportunities in which students collaboratively solve problems, in
the hope of increasing retention of non-traditional computer science students who learn
better with a holistic top-down approach. The goal is to better prepare a diverse group
of students for the work force.
TAILS modules complement an AI survey course. They provide hands-on guided lab
experiments to enhance student ability to develop mental models of concepts. Students
explore software applications that implement algorithms, then, using an application with
portions of code removed, write code needed to implement core components of the
algorithm. Computer science undergraduates in a senior-level Introduction to artificial
intelligence are the primary target for TAILS and its advanced experiments are suitable
for graduate students. The structure of the modules supports use by non-major students
at all levels and provides background for software engineers working in industry.
Industry stakeholders are invited to review and provide feedback on the modules.
Formative assessment will take place in the classroom and in workshops with students
and faculty from multiple institutions.
The project addresses learning outcomes in five categories: skills (ability to solve
problems collaboratively), concepts (demonstrate knowledge of AI concepts and
software engineering practices), communication (describe course concepts at multiple
levels of abstraction), application (identify applications of AI concepts), and research
(demonstrate curiosity about course material). Key deliverables include a collection of
curriculum material, an Internet portal to access all materials, computer science-related
assessments, assessment results, and integration of these into course materials and
classroom activities. The project is entering the beta test and assessment phase.
1
Our approach promotes individual efforts to solve a programming assignment while
building an education community through laboratory work that encourages cooperation
and teamwork among students. The collaborative nature of the experiments reinforces
the social aspect of learning and scientific work with the goal of retaining women in the
major. The paradigm can be adapted to other courses at all academic levels and was
applied to an interdisciplinary animation/computer science course taught in spring 2015.
Hands-on experience with AI algorithms shortens the time required to prepare students
to engage in research. Using AI modules in non-majors courses reaches out to and
potentially recruits from populations not otherwise exposed to such advanced material.
Dissemination has been through conference presentations and posters, and the project
website with additional dissemination activities planned.
Introduction Teaching an introductory course on artificial intelligence (AI) requires a
delicate balance between time spent exploring the intricacies of specific complex
algorithms and exposing students to the variety of algorithms and applications they are
likely to encounter in the future. Given the interdisciplinary nature of most work
assignments, students need to learn how to consider the algorithms from multiple
perspectives and discuss them at multiple levels of abstraction to ensure that all
teammates and stakeholders understand complexities and implications. While all of this
is happening, instructors need to remain mindful of the various learning styles and
interpersonal orientations of all of their students.
The TAILS project1,2 addresses these issues by developing materials and a paradigm
to teach AI by implementing an experiment-based approach modeled after the
laboratory sciences. Students learn about various algorithms by first observing each
algorithm in action, discussing the algorithm in small groups, then experimenting with
changing and observing the results, much as they would in a physics or chemistry lab,
recording their results and observations in a lab report. They work through problems
related to the algorithm collaboratively and in small teams to ensure understanding.
Students develop explanations for what they learn that are geared toward multiple
audiences, and identify real world applications of the algorithms through reflection and
research. The ultimate goal is to produce literate, articulate, students who understand
basic AI concepts and can effectively collaborate to solve problems. The approach is
expected to attract students drawn to collaborative models of working, and increase
participation in the field by exposing undergraduate students to research.
TAILS modules currently cover basic, informed, and adversarial search, agent
architectures, and machine learning with a focus on conceptual clustering. Each module
includes an overview of a concept, sample applications of the concept, and an example
of the algorithm processing input data. The module contains an implementation
This material is based upon work supported by the National Science Foundation under Course,
Curriculum, and Laboratory Improvement (CCLI) Grant No. 0942454. Any opinions, findings and
conclusions or recommendations expressed in this material are those of the author(s) and do not
necessarily reflect the views of the National Science Foundation.
2
independent visualization of the code design represented with UML (unified modeling
language) to provide the big picture of how the code is structured. Students have
access to the corresponding source code to run and modify. Each module is
accompanied by a set of activities that include hands-on exercises that involve problem
sets, experiments, and graduated programming assignments ranging from those
completed with a partner during a lab period to term projects, as well as discussing the
algorithms at multiple levels of abstraction and identifying additional applications of the
algorithms. The majority of class time is spent mentoring students as they explore
concepts with classmates.
For the basic programming exercises, students are provided a version of each program
in which key segments of code have been replaced by comments that guide the
students through the process of implementing the critical component of each algorithm
and testing it with an existing application. Students can access solutions to the
introductory programming experiments. This approach has several benefits. Students
can focus on the AI aspect of the coding without needing to also develop a user
interface. The shorter exercises allow them to gain a sense of accomplishment during
the lab sessions and allow them to learn by comparing their results to those provided.
Critical thinking is nurtured by requiring students to reflect in their lab reports on the
comparison of their solutions to those of other students and to the given solutions.
Students in a senior-level Introduction to artificial intelligence (AI) course are the primary
target of TAILS. The modules include activities for students with a range of
backgrounds, such as those who enroll in an introduction to computer science for nonmajors course or a graduate-level course in AI for systems engineering students with
varying degrees of computer science background.
This paper provides an overview of each of the TAILS modules, followed by a mapping
of learning outcomes to objectives and assessments. It concludes with a brief overview
of lessons learned in the course of developing the TAILS modules.
Basic and Informed Search The basic and informed search modules present a set of
graduated experiments that familiarize students with a variety of search algorithms
including depth-first, breadth-first, non-deterministic search, hill-climbing, best-first,
beam, branch and bound, and A*3. Students study search using
and an eight-city map
of the United States and an application that finds paths through cities on the map as
shown in figure 1. Students select the starting and destination cities and an algorithm to
study, and then observe the way each algorithm searches the map by studying the
search tree produced during the execution of the algorithm. Students reflect on the
differences between the search trees generated to gain an understanding of in terms of
breadth, depth, number of nodes generated, and the length of the final path from the
starting to the destination city. In related exercises students hand draw the tree of all
loop free paths in the state space, and identify the nodes generated and explored by a
specific algorithms. They also analyze the design of the code as reflected in various
UML diagrams and write the lines of code needed to transform the general search
algorithm into one of the target search algorithms. They accomplish this by
3
incorporating into the otherwise complete application the appropriate enqueing
technique. Advanced students are challenged to solve more complex problems.
Figure 1. Basic and informed search application interface.
Adversarial Search The Adversarial Search module builds on Nine Men’s Morris
(NMM), a zero-sum strategy board game1. The board consists of three concentric
squares with the faces of each square connected by four intersecting lines. Each
intersection is a space on the board that can be occupied by one of either player's
pieces. The goal of this turn-based game is to create mills by placing three of the own
player's pieces in adjacent intersections while preventing the opposing player from
doing the same. Students become familiar with the NMM game by playing a version
online with classmates. Next, they play against the NMM computer player agent and
collect statistics on performance to use as baseline in subsequent experiments. Once
familiar with the game and its design, students add alpha beta pruning to the minimax
algorithm and compare performance to the baseline. A more challenging project for
advanced students is to add a state generator to omit symmetrical states from the
generated successor states, or to add a state generator to cache successor states until
branches they belong to are no longer needed.
Agent Architectures The Agent Architecture module of TAILS provides students the
opportunity to learn about agents and their environments in a variation of the commonly
encountered vacuum problem. In this version, lizards are tasked with cleaning up an
4
infestation of insects. The module emphasizes understanding concepts and applications
and presents the idea of agents and environments in the context of a complete software
project. The module is augmented on with a case study of intelligent agents4.
This module presents agents and their environments from three perspectives. First, the
student can select built-in agents, rendered as lizards, and observe and compare their
behavior as they scurry about the floor, represented as a grid of tiles, and devour
insects appearing in the room. Second, students can modify the parameters of the
environment to change whether it is accessible or inaccessible, deterministic or
stochastic, episodic or sequential, and static or dynamic. They can set the size of the
grid, the number of agents acting at a given time, and the number of insects invading
the environment. Students can create obstacles in the environment that the agents must
negotiate. Before attempting to write their own agents, students observe how the agents
behave in a variety of environments and discuss their observations with other students.
Third, students can add agents of their own to the environment and compare their
agents’ behavior to the behavior of the pre-defined agents, or have their agents
compete against those of another student. The user interface for the agent application
displays a mini-map of the area explored by each of four agents set up by the user. As
the agents explore the grid, the application displays a mini-map of the area explored by
each of the agents set up by the user
Conceptual Clustering The conceptual clustering module implements the COBWEB
conceptual clustering algorithm5 and presents a visualization of the taxonomy of the
clusters created from a set of user-specified feature vectors. The internal nodes of the
taxonomy are displayed as objects that reflect the average color and size of the objects
in the subtree below the node, and the most frequently occurring shape. Clicking on a
node reveals the summary data it represents. Students design a data set consisting of
multiple objects described by (color, shape, size) tuples. They experiment with various
input orderings to learn the effect that input ordering has on the resulting taxonomy and
number of high-level clusters that emerge. Students later implement the category utility
function used to determine the cluster to which each new feature vector should be
added.
Outcomes, Objectives, and Assessment Table 1 maps the TAILS learning
outcomes to objectives and assessments. We rely in part of the Dewar-Bennett
Knowledge Expertise Grid6 to analyze our data. The grid defines criteria for summative
evaluation that can be adapted for evaluating knowledge of engineering-related content
and rates a student's affective and cognitive knowledge in terms of the student's level of
expertise.
Lessons Learned and Challenges Ahead TAILS has faced three-fold challenges: 1)
development of engaging, robust, interactive digital learning aids requires a team of
software engineers, content experts, story tellers, and graphic designers9,10,11,12 that
was not envisioned when the project was proposed, resulting in adequate but simple
applications; 2) it is difficult to develop a software application that runs on multiple
platforms over an extended period of time, resulting in the need to periodically revise
5
code; and 3) the extent of the modules developed is broader than can be tested in a
one day workshop and low enrollments in computer science during the last seven years
thwarted efforts to perform classroom assessment. In its place, multiple, partial day
assessment workshops targeting non-majors are planned. Further work born of this
effort includes a proposed workshop to identify best practices to guide those developing
and assessing future online digital learning aids.
Table 1. TAILS learning outcomes [1].
Outcome
Students will
demonstrate the ability
to solve problems
collaboratively
Objective
Student will demonstrate
collaboration and teamwork
skills
Assessment
Students will work in pairs to complete
the lab activities, then:
• Complete a teamwork attitude
questionnaire7
• Write a team process log to record
perceptions about collaboration7
Students will
demonstrate
knowledge of artificial
intelligence concepts
Students will demonstrate
recall and general
understanding of AI concepts
• Answer exam questions
• Complete pre- and post-tests
• Explain and write software code
• Draw a concept map8 p.197-202
• Contrast multiple concepts8 p.168
• Define and give one example of a
course concept8 p.38
• Specify requirements for a software
program
• Complete a domain-level design for a
software program
• Design an algorithm at an
implementation-specific level
• Reverse engineer software for an
algorithm
• Write an elevator statement5 p.183-187
geared toward the student's
grandmother to describe the concept
Research
Application
Communication
Concepts
Skills
Category
Students will
demonstrate
knowledge of software
engineering practices
Students will be able to
describe course
concepts at multiple
levels of abstraction
Students will be able to
identify applications of
AI concepts
Students will
demonstrate curiosity
about course material
Students will demonstrate a
deep understanding of course
concepts
Students will demonstrate
proficiency in software
engineering practices at
background-appropriate
(grade- and majorappropriate) level
Students will be able to
describe course concepts
clearly and without technical
jargon
Students will be able to
describe course concepts for a
classmate or technical
manager
Students will be able to
identify real world applications
for AI concepts beyond those
provided in course materials
Students will demonstrate the
ability to extend course
concepts
6
• Write an algorithm in pseudocode to
describe the concept for a technical
manager
• Complete application cards5 p.236-239
• Describe one new experiment that
can be used in conjunction with each
algorithm studied; explain the
objective of the experiment and why
this is a worthwhile objective
• Describe one enhancement to the
algorithm studied and explain why the
enhancement is worthwhile
References
[1]
S.E. August (2012) "Enhancing expertise, sociability and literacy through teaching artificial
intelligence as a lab science." 119th Conference of the American Society for Engineering Education,
June 10-13, 2012, San Antonio, TX.
[2]
S.E. August, A. Neyer, M.J. Shields, J.I. Vales, and M.L. Hammers. (2010) "Co-opting games and
social media for education." AI and Fun Workshop at the 24th Association for the Advancement of
Artificial Intelligence (AAAI) Conference, July 11–15, 2010, Atlanta, GA; Alelo University seminar
series, Alelo, Inc., August 27, 2010, Los Angeles, CA.
[3]
S.E. August, M.A. Fraser, M.A. Vazquez, M.A. (2014) "Teaching artificial intelligence as a lab
science: basic and informed search" (abstract only). Proceedings of the 45th ACM technical
symposium on computer science education (SIGCSE '14). ACM, New York, NY, USA, 709-709. DOI:
10.1145/2538862.2544273
[4]
S.E. August. (2013) "Tsunami Warning System: a case study of intelligent agents." 27th AAAI
Conference on Artificial Intelligence, 4 th Symposium on Educational Advances in Artificial
Intelligence, July 14–18, 2013, Bellevue, WA. http://modelai.gettysburg.edu/.
[5]
D. Fisher (1987) "Knowledge acquisition via incremental conceptual clustering." Machine Learning,
vol. 2, no. 2, pp. 139-172.
[6]
J. Dewar and C. Bennett. (2004) 8-dimensional Mathematical Knowledge-expertise Grid.
http://myweb.lmu.edu/carnegie/webport/knowgrid.htm, Loyola Marymount University.
[7]
OERL: Online Evaluation Resource Library. http://oerl.sri.com/home.html
[8]
T.A. Angelo and K.P. Cross. (1993) Classroom Assessment Techniques; A Handbook for College
Teachers. 2nd edition. San Francisco: Jossey-Bass.
[9]
S.E. August and J. Ryoo. (co-presenters) (2013) "Virtual environments and game-based learning in
the classroom." National Science Principle Investigators' Forum. Eight meeting online national forum
presented September through November, 2013. http://ccliconference.org/pi-forum/ Refereed.
[10] S.E. August and J. Ryoo. (co-presenters) (2013) "Virtual environments in the classroom." PI-led
Workshop Session C13, 2013 National Science Foundation Transforming Undergraduate Education
in STEM, PI Conference, January 23-25, 2013, Washington, D.C.
[11] S.E. August and J. Ryoo. (2013). Virtual Environments and Game-based Learning in the Classroom.
2013 NSF CCLI/TUES PI Forum. Online: http://security.altoona.psu.edu/gbl/.
[12] S.E. August and J. Ryoo. (2014) "Can 3D virtual world environments and game-based learning
effectively teach computer science concepts?" Proceedings of the 45 th ACM technical symposium
on computer science education (SIGCSE '14). ACM, New York, NY, USA, 737-738. DOI: 10.1145/
2538862.2544264.
7
Biographical Information
Dr. August teaches courses in artificial intelligence, database management systems, software
engineering and introductory programming with Python and Snap! Her research interests include
broadening participation in STEM education using case studies, virtual worlds, and techniques borrowed
from the lab sciences. She also explores applications of artificial intelligence including interdisciplinary
new media applications, natural language understanding, and human-agent-robot teamwork. Dr. August
is actively involved in the Scholarship of Teaching and Learning community and one of the founding
members of CREATE-STEM, an interdisciplinary group of LMU faculty aimed at advancing and promoting
STEM education activities. She is a 2006 CASTL Institute Scholar (Carnegie Academy for the
Scholarship of Teaching and Learning). Dr. August has received NSF DUE IEECI and CCLI grants, and
co-led an NSF PI Forum on Virtual Environments and Game-based Learning in the Classroom. Before
coming to LMU she was a lecturer at LMU, UCLA, and UCI, and a software engineer with the Hughes
Aircraft Company (now Raytheon). Her industry experience includes software and system engineering for
several defense C3I programs and applied artificial intelligence research for military and medical
applications.
Dr. August has served as the Director of Graduate Studies for the Department of Electrical Engineering
and Computer Science at LMU and as the Special Assistant to the Chief Academic Officer for Graduate
Education at LMU. Dr. August is an active member of AAAI, ASEE, ACM, CCSC, and the IEEE Computer
Society.
8