Download Ajax-based Report Pages as Incrementally Rendered Views

The FORWARD
SQL-Based Declarative Framework
for Ajax Applications
Yupeng Fu
Kian Win Ong
Kevin Zhao
Yannis Papakonstantinou
Vicky Papavasiliou
Michalis Petropoulos
Vast Majority of Web Applications:
Issue simple SQL commands, reflect new state on browser
Browser
Submit
Web Application
App server data
in main memory
Session
Session
Session
Objects
Objects
Objects
INSERT
SELECT
UPDATE
Email
DELETE
Paypal charge
Database
2
The Paradox
Most web applications are fundamentally simple;
yet their development requires large amounts of code,
both server-side* and client-side (JavaScript)
*
Java, PHP, C#, Ruby etc.
3
Where is the extraneous code expended?
Low-level code addressing frictions
of the web application architectures
pre-2005: Pure server-side programming
post-2005: Ajax
post-2007: browser went mobile
4
Pre-2005: Pure server-side programming: Code to plumb
across layers; code to optimize data access
HTML
Code with JDBC/SQL calls to
Code to combine
DOM
combine
database
and
JavaScriptdatabase and application
HTML
Browser
Data
access optimization
questions:Components
application-server
data DOM
server data into HTML
-A
single
query
that
gets
both
proposals
&
reviews
Application Server
of reviewer currently in session
- or a single “proposals”Action
query followed by onePage Computation
“reviews” query for each proposal
Request
Java
- or a single “proposals” query Java
with one “reviews”
Parameters
query that lists all relevant proposal id’s
INSERT INTO
SELECT *
reviews …
FROM
proposals…
Session
Session
SELECT *
Session
Objects
Objects
FROM
reviews …
Objects
Database Server
Database
5
2005: Enter Ajax:
A desktop feel to cloud-based applications
• Asynchronous partial update of the page
• Fancy JavaScript/Ajax components
maps, calendars, tabbed windows
6
Ajax: Mundane data coordination between
page state and server state (app server + database)
JavaScript
HTML
JavaScript
JavaScript/Java code
Components
DOM
components
tuned to incrementally
JavaScript/Java
code
have different
update
HTML DOM
& JavaScript
reflects page
data & collect_args();
programmatic
JavaScript
components
response_handler
=
function(){
request parameters
Diffs
partial_renderer();
}interfaces
to the server
send_request();
Browser
Application Server
Request
Parameters
Session
Session
Objects
Objects
Database Server
Action
Page Computation
Java
Java
INSERT INTO
reviews …
SELECT
FROM
SELECT
FROM
Database
Delta
Queries
*
proposals…
*
reviews …
7
For each action, yet another incremental page
computation code and partial renderer
Pure Server-side Model
Browser
Application Server
Action 1
INSERT …
Ajax Model
Partial update
requires different
logic and imperative
programming for
each user action
Partial Renderer 1
Partial Renderer 2
Partial Renderer 3
Action 1
INSERT …
Page Computation 1
Action 2
UPDATE …
Action 2
UPDATE …
Page Computation 2
Action 3
DELETE …
Action 3
DELETE …
Page Computation 3
Page Computation
8
Mobile, iPad & HTML5
• form factor
• Client data & devices; geolocation
• Slower & more unreliable connections
– higher need for client autonomy & low latency
• HTML5
back to web-based standardization
browser-based database
multithreaded asynchronous processing abilities
9
Action and
pageand computations at the browser.
HTML5:
Data
computation
may have integration, coordination exercise
And yet another
enough browser-side data
HTML
to fully avoid server access
DOM
Action
JavaScript
Components
Page Computation
HTML5
Low latency: Browser-side
storage
Browser
computation returns
Application
Server
immediately
available Action
data
Page Computation
instantly, while asynch
requests to the server
Action
Page Computation
initiated for reads or writes
Session
Session
Objects
Objects
Database Server
Action
Database
10
Goal of the
FORWARD Web Application Programming Framework
• Remove architectural frictions & manual optimizations
– Plumbing code addressing language heterogeneities
– Data coordination, optimization code
• Novelty: SQL-based declarative programming
1. “Unified application state” virtual database, comprising
entire application state
accessible via SQL++ distributed queries
2. Application semantics: Action -> Page cycle
3. Ajax page as a rendered SQL++ view that is automatically
and incrementally maintained at each round
Actions and pages can use just SQL++, no Java or JavaScript
• Make easy things easy while difficult things possible
11
FORWARD Application: Set of declarative configurations,
interpreted in a location-transparent way
Browser-side FORWARD JavaScript libraries
JavaScript
HTML
Components
DOM
FORWARD Interpreter (e.g., http://forward.ucsd.edu )
FORWARD Application nsf_reviewing
Action
Page
PageConfiguration
Configuration
ActionConfiguration
Configuration
save_review
review
Source Configurations & SQL++ Schema Definitions
Unified SQL++ Application State
12
FORWARD Application:
Operation
Browser-side FORWARD JavaScript libraries
JavaScript
HTML
Components
DOM
FORWARD Interpreter (e.g., http://forward.ucsd.edu )
FORWARD Application nsf_reviewing
Action
Page
PageConfiguration
Configuration
ActionConfiguration
Configuration
insert-review
review-proposals
Source Configurations & SQL++ Schema Definitions
Unified SQL++ Application State
13
Actions Access Unified Application State via SQL++
Browser-side FORWARD JavaScript libraries
JavaScript
HTML
Components
DOM
FORWARD
Interpreter (e.g., http://forward.ucsd.edu )
SQL++
includes:
Nesting
Core page &
FORWARD Application nsf_reviewing
Variability
request parameters:
Action
ActionConfiguration
Configuration Abstraction
Page
Identity
PageConfiguration
Configuration
of relevant
insert-review
Ordering
part ofreview-proposals
browser state
Source Configurations & SQL++ Schema Definitions
Unified SQL++ Application State
Database
Session
Objects
Action
Outputs
Core Page
Request
Parameters
14
Actions in PL/SQL dialect over the Unified Application State
which access the main memory also (request, session)
// Insert submitted review into reviews table of the hosted database
INSERT INTO hosted.reviews(proposalId, comment, rating)
VALUES (request.pid, request.myComment, request.myRating)
Java
Java
actions
15
Pages as Rendered SQL++ Views over the
Unified Application State
Browser-side FORWARD JavaScript libraries
JavaScript
HTML
Components
DOM
FORWARD Interpreter (e.g., http://forward.ucsd.edu )
FORWARD Application nsf_reviewing
Action
Page
PageConfiguration
Configuration
ActionConfiguration
Configuration
insert-review
review-proposals
Source Configurations & SQL++ Schema Definitions
Unified SQL++ Application State
16
Page Configuration Follows Popular Frameworks:
HTML + FORWARD statements & expressions, using SQL++
<fstmt:for query="
// proposals assigned to
// currently logged-in reviewer
SELECT p.id AS pid, p.title
FROM hosted.proposals AS p
WHERE
EXISTS(
SELECT *
FROM hosted.assignments AS a
WHERE a.proposal = p.id
AND a.reviewer =
session.user">
<tr>
...
<td>{pid}</td>
...
</tr>
</fstmt:for>
Overall structure
reminiscent of XQuery
and SQL/XML
17
Page Configuration: Ajax/JavaScript Components
Simply by FORWARD Unit Tags
<fstmt:for query="
SELECT ...">
<tr>
...
<td>
<funit:bar_chart>
<bars>
<fstmt:for query="SELECT …">
<bar>
<id> {gid} </id>
<value> {score} </value>
</bar>
</fstmt:for>
</bars>
</funit:bar_chart>
</td>
...
After each action, the
FORWARD interpreter
automatically &
incrementally updates
HTML and JavaScript
components
18
Core Page and Request Parameters:
Mirror of named page data
<fstmt:for name="proposals" query="
SELECT
p.id AS pid, p.title
FROM
hosted.proposals AS p
WHERE
...">
<tr>
...
<funit:editor name="myReview">
...
</funit:editor>
...
<funit:select name="myRating“
type=“integer”>
...
</funit:select>
...
<funit:button
onclick="save_review"/>
</tr>
</fstmt:for>
Core Page
proposals
pid
my_review
rating
Automatically
inferred or
explicitly declared
19
FORWARD Demo
20
System Architecture and Optimizations
Browser State
HTML
DOM
JavaScript
Components
1
Visual Page
State
4
Argument Collectors
2
Browser
Partial Renderers
Program Invoker
Incremental
rendering of
JavaScript
components
Server
FORWARD
Interpreter
Program
Email, etc.
Service
INSERT, UPDATE,
DELETE
Services
Unified
Application State
Page Configuration
Visual Units
Page Query in
SQL++
3
Incremental View
Maintenance
Modification Log
Database
State
Session
State
Incremental view
maintenance of data
using modification
log
Visual Page State =
Page
PageQuery + Constants (visual template)
Core Data
State
Request Data
21
Reduction of web programming problems into
data management problems with rich prior work
• Incremental Ajax page maintenance
• Resolving data heterogeneities between the logic visual
layers of the application
• Query optimization for distributed queries over small
main memory sources and a persistent database
• Location transparent code development and low
latency optimizations for mobile
22
Incremental Ajax maintenance
• Use incremental view
maintenance as an efficient
implementation of the page
queries
• Data model features
– Nesting (decorrelation)
– Variability (decorrelation)
– Ordering (ordering attributes)
• Deferred view maintenance
• Large number of small view
instances
• Deletions & updates as
commands that utilize
provenance IDs
HTML +
Javascript State
Partial renderers of
visual units
Page Configuration
Visual Units
Page Query in
SQL++
Incremental View
Maintenance of
Page Query
23
Incremental Rendering of JavaScript Components
BarChartUnit
Insert Bar Chart
new bar_chart();
...
barchart
bar
Tuple
3
value Int
5
Browser State
Delete Bar Chart
Diffs
id Int
Visual units are provided by
framework, and have
renderers that call into
methods of JavaScript
components.
bar_chart.destroy();
...
Replace Bar
Render
HTML
DOM
JavaScript
Components
bar_chart.getBar(id);
bar_chart.replace(...);
...
Find most specific
renderer for unit diff
24
Incremental Rendering of JavaScript Components
BarChartUnit
Insert Bar Chart
new bar_chart();
...
bar
Tuple
Diffs
id Int
3
bar_chart.destroy();
...
value Int
5
Browser State
Delete Bar Chart
Simulate renderers
when necessary
Render
HTML
DOM
JavaScript
Components
25
Resolving heterogeneities between logical and visual aspects
of the page
Report attribute types
and logical structure
dictated by page query
Output of
Page Query
q2c
Input attribute types dictated
by input units
Configuration
Page
c2v
Visual Page
(logical abstraction of
page)
v2c
Types dictated by input
units
c2c
Visually typed
Core Page
Atomic types
dictated by what developer
wants; validation errors
c2t
Typed core
page
t2r
Request
26
Query optimization for distributed queries over small
main memory sources and a persistent database
Replacing the manual specification of data access optimizations that one does
In Ruby-on-Rails, Hibernate
Action (PL/SQL)
Page Configuration
Distributed Queries
Page query
Unified SQL++ Application State
Database
Session
Objects
Action (PL/SQL)
Variables
Core Page
Request
Parameters
27
Related Industrial Work on Removing Cross-Layer Frictions
HTML
DOM
Browser
Application server
Java (C#)
App Server
Objects
Database server
JavaScript
Components
ASP.NET,
GWT
Browser Mirror Java (C#) Objects
?
Database Mirror Java (C#) Objects
Hibernate,
Entity Framework
SQL Data
28
Data management research community efforts: A progression
towards ever more powerful declarative web application specs
• Strudel
– HTML pages as rendered views
• WebML
– From schemas to forms and reports, easily
• Relational transducer
– Hinted that the business logic of many web
applications is captured within SQL
• HILDA
– Unit-based pages & logic
– Client-server computation
29
Future work
• More on algorithms that emulate what “a
good developer would do”
• Client side computation
– enabled by declarativeness
– location transparency
– deliver low latency, disconnected operation
• Optimizations for updating a myriad views
– leveraging pub-sub works
• Visual Do-It-Yourself development
30