Download The system is based on the following fundamental principles:

Developing Dynamic, Database-Driven
Web Pages by Demonstration
David Wolber, Yingfing Su, Yih Tsung Chiang,
University of San Francisco
2130 Fulton St., San Francisco, CA. 94117
(415) 422-6451 [email protected]
ABSTRACT
This paper describes an interactive development
environment for the end-user development of
dynamic web pages that connect to databases.
The system combines techniques from query by
example (QBE), programming by example
(PBE), and the spreadsheet paradigm. In
combination, these techniques allow a designer
to visually specify web pages that display and
manipulate persistent data.
INTRODUCTION
Many HTML WYSIWYG editors allow
designers to create static web pages without
writing code. With the proliferation of dynamic
pages, and specifically those that allow access
and manipulation of server databases, these
editors are not sufficient.
Some recent tools allow the presentation of
dynamic web content to be specified in a similar
way to how static pages are constructed. The
designer creates a template HTML file which
specifies the formatting and layout of sample
data. This template can then be sent to a
translator, such as Enhydra’s XMLC[], which
generates a Java code representation of the
HTML file. A programmer can then manipulate
the content of the page by replacing the sample
data with dynamic data, e.g., data from a
database. This modified HTML is then sent to
the browser.
Tools like XMLC improve the development
process by separating designer and programmer
concerns—designers work on presentation using
the HTML medium, and programmers work on
dynamic content using a programming medium
(e.g., Java).
Unfortunately, such tools do not help designers,
build complete dynamic web pages. They are
still restricted to presentation concerns only, and
must rely on a programmer to implement the
“dynamic” part of the page.
Macromedia's UltraDev 4 (formerly Allaire's
Cold Fusion) is one system that does allow nonprogrammers to visually specify more than just
presentation. UltraDev 4 provides dialogs,
similar to those found in a DBMS, which allow a
designer to map a visual component to a data
source. Without any programming, a designer
can create a complete dynamic web page that
displays live database data. The tool also
provides dialog-based methods of specifying
dynamic web pages that allow end-users to
manipulate that data.
UltraDev4 dialog-based specification eliminates
coding syntax and thus increases the pool of
designers capable of developing dynamic web
pages. But the system does not leverage
important end-user programming techniques that
have been applied in other domains and that can
dramatically ease development compared to
dialogue based specification.
Our research has focused on applying some
known end-user programming techniques to the
domain of dynamic web page development. We
have built a system, DemoWeb, that combines
techniques from query by example (QBE),
programming by example (PBE), and
spreadsheet applications.
Programming in the WYSIWYG Interface
One popular method of end-user programming
involves programming in the interface, a method
made popular in seminal PBE systems such as
SmallStar[], Eager[], and Mondrian[]. These
systems are extensions of macro-recording—the
user tells the system to watch, then demonstrates
the “program” using the same operations that are
used in normal operation of the application. The
system generalizes the demonstrated macros and
creates a program that extends the original
application.
Programming in the WYSIWYG interface is
slightly different. Here, the goal is to create a
brand new program, not extend an existing one.
The designer still “programs” by recording
operations, but he performs this recording in a
facsimile of the “target” interface, i.e., in a
WYSIWYG editor. The operations at his
disposal are not the same ones that will be
available at run-time. Instead, a mapping is
defined from the operations available in the
WYSIWYG editor to operations in the target
application. The designer demonstrates
operations using WYSIWYG editing operations
in order to specify the corresponding operations
in the target application.
One of the most common dynamic components
is the dynamic table. The designer inserts it into
a page just like any static component. The
designer can then use a dialog to map some or all
of its columns to data in a pre-existing database:
The difficulty of this approach is that the
mapping between demonstrated operations target
operations must be clear to the designer—he
must know the semantics of his demonstrations.
But the approach offers some advantages—
because demonstrations are not performed
directly in the target interface, the
demonstrational interface can contain auxiliary
elements to help guide the demonstrations. These
range from labels on visual components that
guide demonstrations, to dialogs that ask for
information from the designer.
Of course the more auxiliary elements provided,
the less WYSIWYG the editor, and the less
concrete the program specification. For instance,
UltraDev4 is based on a WYSIWYG editor
(Dreamweaver), but much of the programming
involves dialogs and other elements
WebDev is a programming in the WYSIWYG
tool that maps HTML editing operations and
dialogue-choice operations to persistent data
operations.
Informal user studies suggest that these
techniques improve the process of developing
dynamic web pages.
OVERVIEW
DemoWeb is based on a WYSIWYG HTML
editor. Designers can use it to create static pages,
or they can add dynamic components to a page,
in which case the system generates a servlet to
render the page at run-time.
Dynamic Components
Dynamic components are HTML visual
components that are mapped to database data.
Alternatively, the designer can enter sample data
into the table, then specify that DemoWeb
should create a new database table with fields
matching the sample data columns (see Section
X).
The granularity of the mapping is not dynamic
table to database table, but dynamic table
column to database table column. Thus, a
dynamic table component can display data from
various database tables. As will be discussed
shortly, it can also contain “formula” columns
not mapped to a database table.
Formatting By Example
When a dynamic table is mapped to a database
table, the system loads a live row of data into the
WYSIWYG editor. The designer then specifies
the formatting of this data by example. The
designer can apply fonts, colors, and other
formatting to the data in each column. At runtime, all of the rows of the database table are
displayed with the formatting the designer has
demonstrated. For instance, if the designer
italicizes the name “Wolber”, as in Figure 1:
Table: STUDENTS
Name
Test1
Test2
Wolber
87
90
the rendered page, created by the system
generated servlet, displays all names in the first
column with italics.
Consider, for instance, a simple form page that
allows the end-user to add a record to an existing
database:
Query By Example
QBE was developed in 1971 [] and some form of
it is now included in most DBMSs. Instead of
requiring the explicit specification of a query, the
system allows the developer to enter sample
data, with an optional relational operator, within
a table cell. The system uses the contextual
information-- the cells where the data is entered-to formulate a formal SQL query. The designer
need not know SQL syntax and is allowed to
work with concrete data instead of abstract
variable names.
In WebDev, the designer specifies queries by
example in the second non-header row of the
dynamic table. The designer enters a boolean
expression (e.g., ">85") in one or more cells.
Table: STUDENTS
Name
Test1
Test2
Average
Wolber
87
90
>85
The system generates an SQL query (in this case,
"Select * From STUDENTS WHERE
TEST2>85") so that, at run-time, only selected
rows of the mapped table are displayed in the
table.
Spreadsheet Functionality
If a column in a dynamic table component is not
mapped to a database, the designer maps it to a
spreadsheet-like formula. The formula can
reference any of the other columns in the row.
For example, a designer could specify
"=AVERAGE(C2:C3)" for column 4:
Table: STUDENTS
Name
Test1
Test2
Average
Wolber
87
90
>85
At run-time, the fourth column of each row will
display the average of the dynamic database data
in the previous two columns of that row.
Programming By Example
The QBE and spreadsheet techniques described
above allow for the development of web-based
database views, but they don't provide a means
for developing pages that manipulate a database.
Figure X. A web page with
UltraDev4 …
DemoWeb provides a PBE technique for
creating such a page. The designer first enters
sample data in the components of the form. The
designer then selects the form's submit button
and chooses Record Response from the
DemoWeb toolbar. Record Response is similar
to the macro recorders in common desktop
applications. However, in this case, the purpose
of recording operations is not so this new
"macro" can be replayed in the development
environment. Instead, the designer is specifying
the operations that should be executed at runtime when a submit button is selected. Also, the
recorded operations will not be executed
verbatim-- the system will infer generalized
operations.
After selecting record mode, the designer enters
new data in the dynamic table component for
which the record should be added. For this
example, let's assume the designer enters data in
the table that matches the data in the form (he
might even copy and paste). When the designer
chooses "Stop Recording", the system compares
the before and after states of the interface to
identify the changes. Since the designer
demonstrated entering the new data in a dynamic
table, the system interprets the operation as a not
just a change to the display, but a change to the
underlying persistent store (database), i.e. a
Table.AddRecord(d1,d2..) operation. And, since
the designer entered data in the table that
matches the data in the form, the system
generates a generalized AddRecord command
=Average(C2:C3)
where d1,d2,etc.
are the names of form elements.
The PBE technique described eliminates the
need for some of the abstraction necessary if
programming or even dialog-based specification
is used. From the designer's perspective, no
name is necessary to refer to the AddRecord
operation, and names are not necessary for the
parameters of that operation (i.e. the names of
the form elements). Instead, the designer
demonstrates the AddRecord operation by
providing an example of what should occur
when the submit button is clicked. The system
performs the pattern matching and generates
code for AddRecord and its parameters. The
designer need only view this code if
modification of it is desired.
Generated Code
WebDemo generates an HTML document and a
Java servlet from the designers WYSIWYG and
visual specifications. The HTML document is
funneled through Enhydra's XMLC compiler.
XMLC creates Java code from an HTML file.
When executed within a servlet, the code renders
the same exact page as if the HTML file was sent
directly to a browser.
Of course the purpose of XMLC is to create a
basis data structure so that a programmer can
easily add dynamic content to the page. The Java
code created by XMLC is based on a tree data
structure fitting the Document Object Model
(DOM) standard. A programmer, or in this case,
the WebDemo code generator, creates code that
manipulates the DOM tree to insert dynamic
content.
WebDemo generates a servlet that contains both
JDBC code to access any mapped databases, and
DOM manipulation code that puts that data into
the HTML sent to the browser.
Servlet
including JDBC and DOM manipulation code
WebDemo Code Generator
HTML
Java DOM code
Figure 3.
Architecture of Generated
Code
For example, if a dynamic table component is
mapped to a database table, WebDemo generates
code like in the following example:
RELATED WORK
WebDemo combines ideas from numerous areas
of computer science: the human-computer
interface field (in particular PBE), the database
field, those studying applications and extensions
to spreadsheets, and those interested in web
development.
PBE
Numerous PBE systems have been developed
and documented in two collections of research
papers[]. One fundamental idea behind this
research is that of programming in the
interface[], that is, allowing the designer to
create programs in the same interface in which
the program is run. Instead of formulating
abstractions like variables, the designer
demonstrates operations on sample data. The
system records the examples then infers or
deduces a general program. PBE has been
applied to numerous applications, including text
and graphics editors[] and spreadsheets. Some
research efforts have also focused on
application-independent PBE.
A number of systems have applied PBE
principles to dynamic web pages, mostly
focusing on the induction of wrapper programs
from examples. A Wrapper program is a web
page that displays data from one or more other
web pages. It generally collects input from the
user, then sends it a set of one or more web
pages, e.g. a number of airline pricing pages. The
wrapper program then identifies the desired data
on the resultant page(s) and displays this
collection of data on the wrapper program page.
Wrapper induction systems eliminate coding by
allowing the designer to point out the pieces of
data in an existing page that should be collected
by the wrapper program. The system then infers
a description of the data, often using contextual
and landmark information so that, if the existing
page changes slightly in the future, the wrapper
will still work.
Server
As far as we know, PBE has not previously been
XMLCapplied to the development of new web pages
that connect directly to databases. However,
examples have been used in databases for years,
as the following section explains.
QBE
Back in 1971, Zloof presented QBE [], a query
by example system. With QBE, the designer
enters data or an expression in a column of a
table, and the system uses the "example" to
formulate a query. For instance, in Figure 1, the
designer has entered "39" in column 2 and ">56"
in column 4. The system generates the query:
Select * from Table Where age>39 and x>56
Some form of QBE is now included in most
Database Management Systems (DBMS). This
idea of example-based database specification has
been extended in numerous research efforts,
including efforts focused on Query by
Templates[] and Triggers by Example[]. It has
not, however, been extended to allow database
manipulation by example, as is the purpose of
WebDev.
Spreadsheet Technology
Spreadsheets are the most popular form of
programming by example. A designer can enter a
formula for a particular cell, then copy that cell
to other cells in the same column (row). The
system then generalizes the example (the copied
cell) and generates formulas that compute data in
the pasted row (column).
Research efforts such as [] have extended the
spreadsheet paradigm to non-tabular interface
components, and have provided more complex
generalization techniques, than that described
above, to handle complex non-tabular
components.
Commercial Web Development
The efficient development of dynamic web pages
is of critical importance to all types of
companies, and new tools are emerging daily to
help the process.
A major concern has been the separation of
designer and programmer concerns. For instance,
in the Java world, servlets alone were first used
to create dynamic pages. With straight servlets,
both the presentation and content of a page are
specified in Java. Presentation must thus be
specified by a programmer, or at least a designer
that is code-savvy enough to enter HTML code
within Java code.
Java Server Pages (JSP) then emerged. JSP
documents are HTML based but allow Java code
to be interlaced with the HTML. JSP eases the
designers task somewhat, but a designer is still
faced with Java code, and the designer and
programmer must still work on the same
document.
Template systems such as WebMacro []
provided the next generation tool. A template
document is stored separately from the Java code
that adds the dynamic content, so designer and
programmer can work on their parts separately.
WebMacro provides a simple scripting language
that allows the designer to mark the places in the
template where dynamic content should be put.
WebMacro library code then processes the script
so that the Java programmer need only associate
code with the names of the placeholders
specified by the designer.
Though WebMacro provides a template, it
doesn't quite allow the designer to format sample
data. The scripting language introduces variables
(any identifier preceded with a '$') and for loops,
which are difficult for some designers. The
designer can supply sample values for the script
variables, however, so a sample page can be
viewed.
Enhydra's XMLC provides what we felt was the
best solution for WebDemo's underlying
mechanism, mainly because it requires no
scripting language other than normal HTML.
With XMLC, the designer creates normal
HTML, including regular tables with sample
data in the cells. To allow the page to be
dynamic, the designer need only provide id tags
for important HTML elements. The XMLC
compiler then parses the HTML and provides
manipulation methods for the identified
elements. The programmer can then easily
modify elements or add new elements, such as
the rows of a table.
With XMLC, the designer need not learn any
special scripting other than HTML. It is also
conducive to WYSIWYG development since
users are comfortable with using property sheets
to name items, but not so comfortable writing for
loops and switching between a visual and nonvisual environment.
UltraDev4-Cold Fusion
Because these emerging template tools are so
new, the visual tools built on top of them are just
now emerging. Cold Fusion, which has now
been integrated into Macromedia's UltraDev 4,
allows much of a dynamic database-intensive
web site to be developed without coding. With
Cold Fusion, a developer uses dialogs to map
graphical tables and other visual components to
pre-defined database tables. UltraDev 4 then
generates server code in several different forms,
including ASP, JSP, and CFML which is a
proprietary scripting language containing SQL
tags.
Design in UltraDev 4 is based on a model-view
paradigm. The designer first maps a data source
to a construct called a RecordSet (the model)
This RecordSet can then be mapped to one or
more visual components.[note that in our system,
model-view is eliminated in design…user works
with the visual components]
UltraDev 4 provides a live data view in which
data from the database is loaded into a visual
component at development time. The normal
view, however, displays variable names in cells
of visual components, e.g., Employee.name.
By default, when a record set is mapped to a
visual component, only the first record in the
record set will be dumped to the component at
run-time. The user must specify a "repeated
region" to have the generated server code dump
all records in the record set.
IMPLEMENTATION
The system was built using the Java II Enterprise
Edition. The Java Swing HTML editing
framework was used for the base HTML editor.
COM is used to access Excel for spreadsheet
evaluation both at development time and from
the server at run-time. The system automatically
registers a developed servlet with BEA’s
WebLogic Server so that it can be executed
immediately during development.
ISSUES
How can QBE applied to interactive operations,
e.g., operations such as the end-user entering
data in a form, then clicking on a submit button
to see a view of a database table dependent on
form data.
In the QBE described in Section x, the designer
specified, by example, a query that determined
the data displayed in a table when a page is first
rendered. But often the data in a table changes
when the user inputs and submits data. How
should the development interface be modified to
allow the specification of such an interactive
behavior.
First, the QBE formulas entered by the user must
be able to reference not only static data or data
from another column, but also data from form
components in the same or other pages. Consider
the example page in Figure X. In the QBE row
of the table, column 3’s QBE formula is
“>text1”. When this page is rendered, the table
will display only those rows of the mapped
database table which have a field3 value greater
than the default value of text1. When the user
enters data in text1 and clicks submit, the
generated servlet will once again select rows,
this time dependent on the new value of text1.
But what if the designer wanted different criteria
depending on whether the page is first being
rendered, or if it being rendered as the result of a
submit. The default value can help—the designer
could give it a very small or very large value to
have the system show all or none of the records,
for instance. But the default value of text1 is also
going to appear in the page, so this is not a very
good solution.
Clearly, the designer needs to be able to specify
a query for the original rendering, and different
queries for the response to interactive
submissions. The PBE technique of recording a
response can be applied here. The designer
selects the desired submit button and selects the
Record Response choice in the toolbar. He then
modifies the desired QBE cell. Because he is
Record Response mode, the system will interpret
the action to mean: when the button is clicked,
repopulate the table with records using the query
defined by the modified QBE cell. When the
designer leaves Record Response mode, the
former value of the QBE cell is restored.
[A different page?]
User Tests/Lessons Learned
Status and Future Work
Summary
REFERENCES