Download Intelligent decision support systems

Intelligent Decision Support Systems
Stephanie Guerlain*, Donald E. Brown*, and Christina Mastrangelo*
-
ABSTRACT In this paper we examine characteristics
common to successful intelligent decision support systems. In
doing this, we attempt to bridge the gap between disparate
communities engaged in building various parts of these
systems. Three systems were examined in detail from widely
different applications and more than 20 additional systems
were considered at a lower level of detail. By examining
deployed decision support systems within the context of a
broad framework we hope to capture the characteristics that
can guide future development efforts. We see this as a first
step in developing an in-depth compendium that will help
bridge the gap between important yet typically isolated fields.
Index Terms -decision support, decision aiding, intelligent
systems, design guidance.
1. INTRODUCTION
This paper explores what it means for a decision support
system to be “intelligent” from a systems perspective.
Several research communities have examined aspects of
intelligent systems behavior, but often these contributions
are merely parts of a solution. For example, many of the
models, algorithms and knowledge-based reasoning
capabilities that have been generated through artificial
intelligence [2] research have led to important contributions
to the intelligent systems approach advocated here. Other
algorithmic work generated out of systems engineering
research, such as data mining, data fusion, decision analysis
[ 121, and optimization techniques [IO] have also
contributed greatly. However, even though these
communities have explored means for making computer
systems exhibit intelligent behavior, many of these systems
have failed in use due to their brittleness [l], [4], [14],
complexity [ 191, and poor interfaces to other systems and to
the people having to use them.
Making a single machine act intelligently may be much less
useful or important than being able to cooperate in an
environment with other intelligent agents, whether they are
humans or machines. Beyond being able to act intelligently
in isolation, it is necessary for a system to be able to
communicate with others, detect and correct mistakes, and
take advantage of others’ abilities, so that overall
“intelligence” or effectiveness may be an emergent property
of all the smart agents working on the problem in a
relatively coordinated fashion. With this spirit in mind, we
put forth several properties that together make up the
characteristics of an intelligent system. Although not
exclusive or exhaustive, we hope that this list serves a
useful purpose in the design itnd evaluation of intelligent
systems that truly act as “team players” [l I].
11. METHOD
Our method was to identify those decision support systems
(DSSs) that are successful in practice and to identify and to
classify those domain-independent characteristics that were
present in the majority of these systems. Briefly, we looked
at:
The Antibody IdentlJicar‘ionAssistant (AIDA), also
adapted as the Transfusion Medicine Tutor (TMT) - a
DSS that uses an expert knowledge base to critique
blood bank practitioner!; as they perform a medical
laboratory analysis task [7]. This system is currently
in use as an intelligent tutor for medical technologists
throughout the country [151.
The Regional Crime Analysis Program (RECAP) - a
DSS that mines large crime databases and uses
intelligent clustering techniques and other statistical
techniques combined with a geographic information
system to help crime analysts to discover patterns in
the data [I]. This system is in use by law enforcement
in several cities.
A River Flooding Forecasting System - a DSS that uses
probabilistic (Bayesian) reasoning to predict the
likelihood of river flooding based on weather forecasts
[9]. This system is in use by the National Weather
Service.
In addition to looking at thlese systems in detail, our
intelligent systems characteristics are supplemented by the
literature on successful dec.ision support system design and
our extensive collective experience in developing and
evaluating such systems.
111. NEW ASPECTS OF THE WORK
‘Department of Systems Engineering
University of Virginia
Charlottesville, VA 22904-4747
0-7003-6503-6/001$10.0Q 0 2000 IEEE
Our eventual goal with thi,sresearch is to develop a
compendium of design features, algorithmic techniques,
and other important characteristics, and to map domain
1934
properties into these effective design techniques and vice
versa. For example, aplanning domain might have the
design feature of being able to conduct “ What-If’searches”.
The compendium will point to previous systems that have
implemented this What-If capability, and describe the
relevant requirements for such a feature to work, from the
data management requirements to the algorithmic
requirements, to the human-interaction requirements. This
compendium will characterize and classify domain
properties and design features in several different ways.
This will serve as an invaluable resource for developers of
decision support systems, enabling designers to conduct
searches on techniques used by others and avoid having to
reinvent the wheel each time a new system is developed.
IV. RESULTS
Successful DSS’ and their subsystems must act intelligently
and cooperatively in a complex domain with potentially
high data rates and make judgments that model the very
best human technicians. It is also crucial that human
technicians maintain control over the final judgments, either
by focusing the system on particular reasoning goals, or by
modifying the basic knowledge on which the system’s
judgments rely. Our work so far has identified the
following attributes that represent high-level characteristics
of successful systems:
A. Znteractivity:the system works well with other
databases and with the human users who must work
with it and allows those agents to explore the “space of
possibilities” in a constraint-based way, instead of just
providing the one “optimal” solution.
B. Event and Change Detection:the system recognizes
and effectively communicates important changes and
events.
C. Representation aiding: the system represents and
communicates information in a compelling,
informative, and human-centered way.
D.Error Detection and Recovery: the system checks
for typical reasoning errors made by people. Further,
the system has some knowledge of its own limitations
and checks for situations for which it may not be as
fully capable.
E. Information out of Data: the system uses intelligent
algorithmic techniques to massage data and generate
information. This means both extracting information
from large amounts of data as well as providing the
tools for handling small data sets, outliers, and other
sources of error and confusion.
F. Predictive Capabilities: The system can predict the
effect of actions on future performance. Notice that
this means predicting both the future environmental
state in addition to the change in states caused by
different decisions.
Each of these characteristics will be described in more
detail below.
A. Interactivity
The first requirement is that the system works well with
others - interfaces well with other databases and with the
human users who must work with it. The system does not
just advocate the “one best way”, but allows users and other
agents to explore the space of possibilities instead of just
providing the one “optimal” solution. That way, the same
smart algorithm that is used to generate the optimal plan
can also be used to evaluate user-generated plans, or to
generate a different optimal plan based on tweaked inputs
to the system. This approach is used in all three of the
systems analyzed above as well as in several research
prototypes, including: an advisory system for commercial
flight planning to route planes around bad weather [ 141, in
an advisory system developed for building operator
supervisors who must decide when to charge and discharge
a thermal energy storage system [SI,as well as in a display
designed to support operators controlling a thermodynamic
process [ 161.
B. Event and Change Detection:
This characteristic monitors the on-going system and
signals to the user when a DSS parameter(s) has changed.
Ideally, it is proactive in nature. In other words,
appropriate system parameters are tracked over time such
that a trend or step change is observed prior to a system
catastrophe. In its simplest form, this concept is analogous
to statistical process control. This technique monitors the
signal and the variance of the signal of key system
parameters andor surrogate variables. When the system is
stable, a certain level of randomness is acceptable. An
example of this concept can be seen in RECAP in that a
type of crime may be monitored over time. An increase in
this crime would be automatically detected if the number of
occurrences or rate of occurrence exceeded either a
statistically-determined or user-specified limit; the system
alerts the user and action may be taken before a hot spot
becomes out of hand. This decision support technique is
seen in many successful systems, and was the explicit focus
in a tool recently developed in the petrochemical refining
industry [6).
C. Representation Aiding
All successful decision support systems represent and
communicate information in a compelling, informative,
human-centered way. The river flood forecasting system is
so successful because it takes complex, probabilistic data
and uses easy-to-understand graphical techniques that
enable the general population to understand the likelihood
and implication of river flooding in their local district.
Techniques such as providing an overview display,
representing domain properties through appropriate
graphical properties, and highlighting important changes
and events are used in this and other successful decision
1935
support systems, e.g., DURESS [ 161 and the MPC
Elucidator [8].
D.Error Detection and Recovery
The system has some knowledge of its own limitations and
can check for situations for which it may not be as fully
capable. Further, the system can check for typical
reasoning errors made by people. This can be done in
several different ways, either through representing
uncertainty directly, as is done in [9] or by having a
knowledge-based expert system monitor for conditions that
the system is not capable of handling, as is done in [4].
E. hformation out of Data
Successful decision support systems help people make
sense out of increasingly enormous amounts of data. Data
collection is becoming ubiquitous, and anyone trying to
analyze such data can be overwhelmed. For example, in the
military, science, and medicine new sensor systems provide
increasing amounts of data to decision makers. These
sensors operate over an astounding amount of the
electromagnetic spectrum. Unfortunately, these sensors are
typically designed by different companies or result from
different government programs. This results in a jumble of
data from different sources that needs to be integrated to
provide real information to decision makers.
Intelligent algorithms can help turn data into information.
Two technologies discussed here are data fusion and data
mining.
Data fusion organizes, combines, and interprets information
from multiple sources, and it overcomes confusion from
conflicting reports and cluttered or noisy backgrounds.
Data fusion is essential for management control over a wide
range of operations and for strategic planning. As an
example, military operations in areas such as Kosovo or
central Africa require the benefits of data fusion systems to
combine and make sense of data about hostile forces,
friendly forces, neutrals, unarmed civilians, and
environmental factors. In a similar manner, large industrial
and commercial concerns now need data fusion techniques
to manage operations over large geographic distances and
across a variety of products and services. Table 1 shows
data fusion systems from the military domain that we have
investigated.
Data mining is concerned with the automatic discovery of
patterns and relationships in large databases. These patterns
can suggest ways to improve processes, reduce costs,
eliminate errors, and generally streamline operations. Data
mining requires that enterprises consolidate data from each
of their processes in data warehouses or data marts. This
can be an expensive undertaking for enterprises that have
grown with separate information systems supporting
separate processes and organizations. However, the
improvements resulting from access to a consolidated view
of the enterprise made possible by the data warehouse can
make this work worth the expense. Data marts represent
intermediate data warehouses that consolidate data for
subsets of the enterprise. As such they can provide more
immediate but less encompassing access to data within the
organization. The ReCAP syistem [ 11 is an example of a
decision support system using data mining on a regional
data warehouse of crime data.
Techniques within both of these areas, data fusion and
data mining, provide the means to generate relevant higherorder information, represent iind manage uncertainty, and
detect important changes and events. Again, most all of the
DSS systems discussed so far have intelligent algorithms
designed to create information out of data, although most
work with something less than complete enterprise level
data.
Program
ASAS
IOrganization
Objective
Amy
All source correlation
and fusion
Navy
Data processing for
JMCIS
command and control
Air Force
Correlation for tactical
TBMCS
operations
Marines
Operational support
IAS
SPAWAR
Common operational
GCCS-M
picture, data fusion
tools
DDB
DARF'A
Real-time multisensor fusion for wide
area tactical decision
support
Wargoddess NSA
Strategic and
operational support
Hercules
BMDO
Missile defense
CEC
NAVSEA
Command and
operational support
Operational suppog
Analytical support
Table 1. U.S. Data Fusion Programs
F. Predictive Capabilities
Many but not all successful decision support systems have
some predictive capability. Within this characteristic we
include both strategic and tactical or operational
capabilities. Strategic prediction involves assessing
impacts from major changes both within and exogenous to
an enterprise. For example, predicting the impact from
building a new plant, relocating a service facility, or
reacting to a competitor's entry into a new market.
Tactical prediction involves changes in operations and
typically looks out into the near future. This type of
prediction supports activities like scheduling, purchasing,
and inventory management for commercial enterprises. For
govemmental agencies tactical prediction supports similar
1936
'
activities but also includes emergency operations and
service provisioning.
Both strategic and tactical prediction can involve predicting
outcomes from decision making. This entails techniques
such as “what-if” analysis, simulation, gaming and the like.
It also requires learning about relationships between
variables, and initiating corrective actions as necessary so
as to predict the effect of actions on future performance.
Both the river forecasting and the crime analysis package as
well as the Flight Planning Testbed [ 141 and the Thermal
Energy Storage Advisor [5] use prediction capabilities in
providing decision support.
171
In addition to prediction of environmental system state, the
system can also predict user and system states. For
example, it may predict both changes to user preferences
and also changes to its own internal state as new
information becomes available. In order to do this, it must
be able to specify desired (nominal) performance levels and
then show how these can change. In its design, the
Wargoddess system mentioned in Table 1 employs a system
control approach that selects algorithms based on their
expected performance with arriving sensor data. In this
sense, it predicts both the performance of the algorithms
and the expected environmental states over the period of
performance. We believe this type of control will become
increasingly evident and important in intelligent decision
systems.
[91
V. CONCLUSIONS
This paper represents a first step in classifying
characteristics that make an intelligent decision support
system. We are using a broad framework to include the
contributions from many diverse fields, such as artificial
intelligence, data mining, data warehousing, and cognitive
systems engineering. By examining deployed decision
support systems we hope to capture the characteristics that
can guide future development efforts. We see this as a first
step in developing an in-depth compendium that will help
bridge the gap between these important yet typically
isolated fields.
VI. REFERENCES
[ 11
[2]
[3]
D. Brown, “The regional crime analysis program
(RECAP): A framework for mining data to catch
criminals,” presented at IEEE Conference on
Systems, Man, and Cybernetics, 1998.
M. Firebaugh, Artificial Intelligence. A KnowledgeBased Approach. Boston: PWS-Kent Publishing
Company, 1988.
D. Gregory, “Delimiting expert systems,” IEEE
Transactions on Systems, Man, and Cybernetics, vol.
SMC-16, pp. 834-843, 1986.
1937
S. Guerlain, “Using the critiquing approach to cope
with brittle expert systems,” presented at
Proceedings of the 1995 Annual Meeting of the
Human Factors and Ergonomics Society, 1995.
S. Guerlain, “Interactive advisory systems,”
presented at Human Performance, Situation
Awareness and Automation (HPSAA) Conference,
Savannah, GA, 2000.
S. Guerlain and P. Bullemer, “User-initiated
notification: A concept for aiding the monitoring
activities of process control operators,” Proceedings
of the 1996 Annual Meeting of the Human Factors
and Ergonomics Society, Philadelphia, PA, 1996.
S. Guerlain, P. J. Smith, J. Obradovich, J. W. Smith,
S. Rudmann, and P. Strohm, “The Antibody
Identification Assistant (AIDA), An example of a
cooperative computer support system,” presented at
Proceedings of the IEEE International Conference on
Systems, Man, and Cybernetics, Vancouver, BC,
1995.
S. Guerlain, G. Jamieson, and P. Bullemer,
“Visualizing model-based predictive controllers,”
presented at Proceedings of the Human Factors and
Ergonomics Society 44th Annual Meeting, 2000.
L. Gunderson and R. Krzysztofowicz, “Interface for
displaying probabilistic river forecasts to decision
makers,” presented at 14th Conference on
Hydrology, 1999.
F. Hillier and G. Lieberman, Introduction to
Operations Research, Fourth Edition. Oakland, CA:
Holden-Day, Inc., 1986.
J. Malin, D. Schreckenghost, D. Woods, S. Potter, L.
Johannesen, M. Holloway, and K. Forbus, “Making
intelligent systems team players: Case studies and
design issues,” NASA, Technical Memorandum
104738, 1991.
H. Raiffa, Decision Analysis: Introductory Lectures
on Choices Under Uncertainty.New York: Random
House, 1968.
S. Potter, D. Woods, T. Hill, R. Boyer, and W.
Morris, “Visualization of dynamic processes:
Function-based displays for human-intelligent
system interaction,” presented at Proceedings of the
1992 IEEE International Conference on Systems,
Man, and Cybernetics, Chicago, IL, 1992.
P. Smith, E. McCoy, and C. Layton, “Brittleness in
the design of cooperative problem-solving systems:
The effects on user performance,” IEEE
Transactions on Systems, Man, and Cybernetics-Part
A: Systems and Humans, vol. 27, pp. 360-37 1, 1997.
P. Smith, J. Obradovich, S . Guerlain, S. Rudmann, P.
Strohm, J. Smith, J. Svirbely, and L. Sachs, “Design
concepts underlying the use of an expert system to
teach diagnostic reasoning for antibody
identification,” presented at IEEE Conference on
Systems, Man, and Cybernetics, San Diego, CA,
1999.
[ 161 K. Vicente, K. Christoffersen, and A. Pereklita,
“Supporting operator problem solving through
ecological interface design,” IEEE Transactions on
Systems, Man, and Cybernetics, vol. 25, pp. 529-545,
1995.
[ 171 K. Vicente, Cognitive WorkAnalysis. Toward Safe,
Productive, and Healthy Computer-Based Work.
Mahwah, New Jersey: Lawrence Erlbaum
Associates, 1999.
[ 181 R. Will, “True and false:dependence on technology:
Evaluation with an expert system,” Computers in
Human Behavior, vol. 7, pp. 171-183, 1991.
[ 191 D. Woods, ‘‘Decomposing automation: apparent
simplicity, real complexity,” in Automation and
Human Performance: Theory and Applications, R.
1930
Parasuraman and M. Mouloua, Eds. Mahwah, NJ:
Lawrence Erlbaum Associates, 1996, pp. 1-16.