Download AI Research to AI Business, and Back: Automatic Story Generation

WC Original – 4,248
AI Research to AI Business, and Back: Automatic Story Generation and
Intelligent Document Production1
Selmer Bringsjord, PhD
I toiled for years (with Dave Ferrucci of IBM's T.J. Watson Research Center)
attempting to build an AI system capable of writing (short short) stories with at
least some literary virtue. The fruit of this basic research, to this point, is the
BRUTUS system, described in the recently published Artificial Intelligence and
Literary Creativity: Inside the Mind of BRUTUS, A Storytelling Machine
(Bringsjord & Ferrucci, 2000). To some people, work on BRUTUS seemed
irremediably abstract and impractical. However, we describe here the efficacy of
a BRUTUS approach to Intelligent Document Production (IDP) for the business
world. IDP is the core business of Document Development Corporation (DDC),
and includes AI software that collaborates with humans to create and produce
documents of any type. It turns out that IDP solves two difficult problems
plaguing both business and AI. Here AI research feeds business, and business
software in turn solves research problems.
Introduction
What comes first, business innovation and profit, or basic research? This is not
an easy question to answer. In our opinion, basic research comes first, and
makes business possible. However, business does not simply apply research for
profit: Sometimes business advances what research has started.
We performed years of basic research in an attempt to build an AI system
capable of writing (very short) stories with some literary virtue. To conceptualize
this goal, imagine building a system that successfully competes against humans
in the Short Short Story Game (S3G). Alan Turing (1964), a grandfather of AI and
computer science, introduced a landmark imitation game, the Turing Test (TT). In
this test, a computer is in one room, a person in another. A judge, not knowing
which room holds the computer and which the person, must ascertain, solely
through electronic questions and answers, which room holds which player.
Turing held that when a judge does no better than chance when delivering
verdicts as to which room contains which contestant, a true “thinking” machine
will have arrived. He predicted that such thinking computers would be amongst
us by the turn of the century.
Polymath and genius though he was, Turing was no prophet: The celebratory
pyrotechnics of 1999 New Year's Eve have now long faded, and Turing's
prediction has failed to materialize. On the other hand, few doubt that computers
will eventually pass the TT. This may not be true for S3G, which is similar to TT
but specialized to stories. In S3G, the human and computer are again hidden
from the judge. However, the judge emails both players a single sentence, and
then waits for the output from both, which must come in the form of a short short
story (< 500) words. The judge determines which story is better, from the
standpoint of literature.
Short Short Story Game
Bringsjord and Ferrucci claim that when a computer prevails in S 3G against, say,
John Updike, as Deep Blue did against Kasparov, then we will have AI worth
writing home about (See Figure 1). An S3G-passing system is a long way off
(Bringsjord, 1998). However, Bringsjord and Ferrucci produced BRUTUS, and
though this system is not a silicon Shakespeare, it manages to generate simple
stories. Indeed, in an advertised contest, AOL posted one such story along with
some human-authored ones, and asked members to determine which story was
machine-authored. Surprisingly, the humans in general could not tell which was
which. Perhaps the main reason why BRUTUS did so well is it ran on the
strength of robust knowledge about literary themes. For example, BRUTUS
contains a formal, knowledge-based definition of betrayal. It was also
monumentally helpful that BRUTUS was not asked to write a story on the fly,
based on one wide-open sentence.
Figure 1: The Short Short Story Game,
or S3G for Short.
If you think anyone who builds things such as BRUTUS just enjoys grappling with
purely theoretical questions -- you are right! Its creators are intensely curious
about how far logic can reach into the supposedly non-logical realm of literary
creativity. All the relevant AI R&D is currently based on logicist AI, as described
in (Bringsjord & Ferrucci, 1998). This curiosity was presented in the early days of
story generation research and embryonic versions of BRUTUS, Since it was
explained to audiences in the context of logic programming intricacies and
knowledge-based AI, BRUTUS's creators nonetheless appeared to be
quintessentially detached.
More than a few business-oriented people, hearing a lecture and seeing a
demonstration of BRUTUS (in which a stunningly simple story scrolled out across
the screen), expressed utter incredulity that two apparently bright guys would
spend large amounts of time in searching for computer programs able to
generate excruciatingly simple stories. As more than one audience member
asked: “What can this technology do?" The answer is that ivory-tower curiosity is
only part of the story. The other part involves what we see as an axiom: If basic
research is rigorous and general, there will be money and profits at the end of the
research.
The Computer Story
Before there were physical computers, there were mathematical computers
dancing in the head of Alan Turing. In his immortal “On Computable Numbers
with Applications to the Entscheidung-Problem" (1936), Turing introduced the
notion of a programmable computer. These machines, now known as (universal)
Turing Machines, were non-physical -- only mental constructs. Turing seemed to
just toy with his mental conception. Indeed, in a little-known but fascinating
chapter in AI’s history, he played chess as a computer against a human on BBC
radio, by simulating his conceptual account of computation. For a good long
while after inventing the machines that bear his name, Turing did not think that a
physical programmable computer would actually exist: he thought physicalized
TMs would simply be too large to be real.But because Turing's conceptions and
methods were both general and rigorous, his basic research led tocomputers as
we know them today: physical artifacts that solve concrete problems.
The situation was similar with AI. Newell and Simon showed up at the famous
1956 Dartmouth workshop where AI officially started (where John McCarthy
coined the phrase “Artificial Intelligence') and stole the show -- with their LOGIC
THEORIST system. This system solved abstract yet limited problems. For
example, LOGIC THEORIST could prove that if given “p then q”, it could be
transposed to the equivalent “if not-q then not-p”. However, Newell and Simon,
similar to Turing, had taken an approach that was general and rigorous. Their
approach, automated reasoning in first-order logic, anchors modern-day AI. This
is seen in the role-played by first-order logic in one of the dominant AI textbook of
today: Russell and Norvig's Artificial Intelligence: A Modern Approach (Russell &
Norvig, 1994); and witness, below, the role played by this logic in DDC's own
systems.
Business Profits
So where is the profit from this research? What good is the BRUTUS type
technology to business? The answer is really quite simple. Systems based on
logic programming and knowledge-based AI can automate the creation and
production of sophisticated business documents. Intelligent Document
Production (IDP) systems query humans for key input, and then automatically
generate business documents -- proposals, contracts, memoranda of
understanding, etc. This generation is not simple variable binding. These
systems generate business documents based on formally represented
knowledge and rules, just as BRUTUS generates stories. For example,
documents that drive the IT Services industry use formal knowledge, which
captures the extensive and intricate human knowledge experts have regarding
the structure of engagements in that industry.
The economic value of Intelligent Document Production systems lies in the fact
that they automate document production. Automation can mean real value, and
real profit. IDP automates the creation, production, and management of
documents -- something that is ubiquitous and yet in demand (without documents
business is impossible).
Two Problems
Two big problems, however,confront IDP as described so far. The first is that
creating effective business documents is an intensely iterative process. While in
the literary sphere it may be sufficient for an author to deliver a manuscript that
no one else modified or edited, in the business world allimportant documents are
rewritten, restructured, polished, refined – often with a team of people. Even after
multiple iterations, this process is likely to start all over again. In light of this, even
a system that automatically generates a first draft based on a robust knowledge
base, and intricate rules and constraints, is not of great valuae in the corporate
world. AI is still struggling to solve the problem of building a system that allows
continuous interaction with multiple human authors to construct a finished
document. One reason is that for this problem to be solved, the computer author
must be aware of changes made by human co-authors, and such machine
awareness, even serviceable approximations of it, has proven elusive. Even if
such machine awareness is reached, there remains the associated problem of
integrating perceived human behavior with the underlying logic of document
production.
The second problem is one that has slowed wide deployment of expert systems
in our economy – retrieving all of the pertinent knowledge from an expert and
getting it into a machine. This process is difficult and so time-consuming that it is
not always practical.
Along with others at DDC, have developed IDP software that solves these two
problems. See the side bar The Scenario for an example of how this technology
would work on a document.
How Do IDP’s Work?
Obviously, we cannot present the complete details in this article for two reasons.
The first reason is that these systems are very technical in nature, involving
advanced logic programming-based software systems – not something explained
in a relatively short article. The second reason is that the company, DDC, has
potential competitors and we would prefer not to give away our technology.3
However, we can characterize the overall approach in terms all readers can
grasp in a short article.
To understand this characterization, you must understand that good AI must be
based on explicit, structured knowledge representation and machine reasoning
(Laird, 2000), with humans engineering that knowledge and reasoning. DDC's
technology, based upon years of working within the paradigm of so-called
“logicist" (= ``symbolicist" ) AI, has seen nearly $5 million go into R&D. We now
provide an idealized account of these systems as harnessing the power of
deduction and proof, cornerstones of logicist AI.
We need, first, a brief encapsulation of first-order logic (FOL). FOL's alphabet
contains variables,
, constants
, n-ary relation symbols
(and various other uppercase Roman letters chosen to be
mnemonics), functors
(and various other lowercase Roman letters
chosen to be mnemonics), quantifiers
, and the familiar truth-functional
connectives ( “not" in English), ( “or"), ( “and"),
( “if
then
"), and
( “
if and only if
").
Variables work in FOL in a similar fashion to how they work in elementary
algebra, except that while variables in algebra range over numbers, variables in
FOL range over literally any object class. Constants can be thought of as names
for particular objects. Relation symbols represent properties objects can have.
Functors represent functions. The quantifier can be read “there exists an
"
and can be read “for all
." Here is a simple formula in FOL that might be
interpreted as saying that all readers of PCAI are smart:
Now here is a formula in FOL that uses more of FOL notation:
This formula is intended to say that if x is a first name for someone, and y a last
name, then the string constructed by concatenating x, a space (denoted by the
constant ), and y, is a complete name suitable to appear in some document.
So, if Selmer is a first name (i.e., F(Selmer), and Bringsjord is a last name (i.e.,
L(Bringsjord), the function designated by s, taking these constants as argument,
produces the string "Selmer Bringsjord", and a machine can generate a proof
that this string is a complete name, i.e., that C is true of it. The proof can be done
in two steps. In the first step, known in mathematical logic and computer science
as universal elimination, the formula above has its universal quantifiers removed,
and the variables associated with them changed to relevant constants. So we
can deduce:
If now we work from the inside out, and let the function s do its work, we reason
to
Given that the facts
and
are in the system's
knowledge base, the forward chaining rule modus ponens, which says that from
and
,
can be detached, allows the system to deduce
At this point, you have to put your thinking cap on. Imagine that a document, or a
50-page proposal, is the conclusion of a proof. Imagine that all the domain
knowledge for a particular class of documents, automatically harvested from the
minds of relevant humans, is represented in some FOL-based knowledge base
from which the conclusion in question is proved. Imagine as well that we place
constraints on how certain proofs are carried out, and on what is proved, given
the knowledge base. In addition, imagine that these constraints, presented in
intuitive, easy-to-grasp form, are adjusted by humans working interactively with
the system. What you have imagined, we have built. The entire system works
blindingly fast, and someone without the slightest bit of technical knowledge can
effortlessly pilot it.
Bibliography
Bringsjord, S. (1998),
`Chess is too easy', Technology Review 101(2), 23-28.
Bringsjord, S. & Ferrucci, D. ( 1998),
`Logic and artificial intelligence: Divorced, still married, separated...?', Minds and
Machines 8, 273-308.
Bringsjord, S. & Ferrucci, D. ( 2000),
Artificial Intelligence and Literary Creativity: Inside the Mind of Brutus, a
Storytelling Machine, Lawrence Erlbaum, Mahwah, NJ.
Laird, J. (2000),
`Bridging the gap between developers and researchers', Game Developer
7(8), 34.
Russell, S. & Norvig, P. (1994),
Artificial Intelligence: A Modern Approach, Prentice Hall, Saddle River, NJ.
Turing, A. (1964),
Computing machinery and intelligence, in A. R. Anderson, ed., `Minds and
Machines', Prentice-Hall, Englewood Cliffs, NJ, pp. 4-30.
Turing, A. M. (1936),
`On computable numbers with applications to the entscheidung-problem',
Proceedings of the London Mathematical Society 42, 230-265.
BIO:
Selmer Bringsjord, PhD is the Chief Scientist at Document Development
Corporation and Director, Minds & Minds Machines Laboratory Professor of
Logic & Cognitive Science Rensselaer Polytechnic Institute (RPI) Troy NY. He
can be contacted at [email protected].
Side Bar
Figure 2: A Member of the Team Emphatically Goes with Time and Materials
Figure 3: A Shift to Fixed Price
Figure 4: System Warns of Change
Figure 5: The Customization is Displayed
A Scenario
The IDP system works interactively with human authors to produce documents
that meet the requirements. We demonstrate this, by walking through a
sequence of interaction, in which the IDP exploits not just its pre-installed
knowledge of documents within a given domain (which here happens to be IT
Services), but also its ability to detect what human co-authors are doing, and its
ability to take intelligent action given what it has sensed.
We begin with Figure 2, which shows a proposal for an IT Services engagement.
A member of the team has typed under “3.8 Estimated Project Billing" that the
engagement will be conducted on a time and materials basis only. We assume
that this is because this form of charge is generally preferable.
However, if the deal is in jeopardy, another team member could inform the IDP
that labor charges need to be fixed price-based -- generally regarded preferable
from the customer's point of view (Figure 3). At this point the IDP, informs all
users that this new basis is inconsistent with prior customization (time and
materials - see Figure 4). In Figure 5, we see that IDP is aware of just which
customization leads to a conflict: the customization in question is displayed.
Figure 6 is a snapshot indicating IDP’s explanation for the inconsistency, and a
suggestion for how the problem can be quickly addressed (which involves
choosing the appropriate component for the relevant variable). Once a member
of the team selects this component, the system goes into autodraft mode, and a
new document is generated (see Figure 7).
Figure 6: Explanation of Violation
Figure 7: New Document is Generated
Capturing The Expertise
How does the IDP capture the expertise – the 2nd problem? In general, by turning
to automation, and immediately using the interactivity that solved the first
problem. The problem of knowledge extraction arose in the first place from
attempts to automate (by extracting knowledge from humans and imparting it to
computers armed with the reasoning ability), and so by turning back to
automation. In automating the process of knowledge engineering for IDP, DDC
did have one advantage others who have worked on solving the “extraction
bottleneck" didn't have: documents offer clues to what those humans who
produced them were thinking. To clarify this, we demonstrate automatic
knowledge extraction, working in conjunction with humans.
Figure 8: A Legacy Document With Naive Use of Variables
We start with a legacy IT document, a General Agreement to be specific. Let's
assume, in addition, that this starting document does have some naive variables
as shown in Figure 8. The first step is to import the legacy document into another
IDP, a five-step process that we will truncate here to fit within the space we have
(see Figure 9). assuming that the outlining parameters for parsing the legacy
document have been determined by the user(s), we confirm and import the
document. The result is a “smart" document that has not only been outlined, but
carved up into components that can be moved, switched with other components,
deleted, hidden, and so on (see Figure 10). (Of course, these actions can be
carried out not only by human authors, but also automatically by the IDP).
The next phase converts parts of the smart document into variables over which
tools can reason; this conversion is the job of another IDP, collaboratively with
the relevant humans. The process involves four substantive steps, and an
introduction (see Figure 11). Let's assume that the users opt to have monetary
values and fixed words not in the (spell checker) dictionary converted to
variables, and that they also elect to have the characters < and > used as flags in
accordance with the naive variables present in the original legacy document (see
Figures 12 & 13). At this point, as shown in Figure 14, the user is presented with
a list of candidate variables, and after making a selection, the system completes
the process. The end result, indicated by Figure 15, is a rather smart, interactive,
“meta-document" and associated knowledge, which together are sufficient now to
allow humans to rapidly create and produce, via the IDP documents for a wide
array of IT engagements.2
Figure 9: The Five-Step Process
Figure 10: The Document is Outlined and Carved Up into Flexible Components
Figure 11: The Steps Involved
Figure 12: Some Constraints on Variable Creation
Figure 13: More Steps on Variable Creation
Figure 14: Candidate Variables Shown
Figure 15: Result: A Smart, Interactive, ``Meta"-Document