Download Semantics in Declarative System

Semantics in Declarative System
The Evolution of Business Unit Empowerment
M
D
Metadata Solutions
Dan McCreary
Dan McCreary & Associates
Wednesday, 5/23/2007
8:00 AM - 9:00 AM
Level: Business/Strategic
Presentation Summary
“Declarative programming” has become the latest
buzzword to describe languages that abstractly
define systems requirements (the what) and
leave the implementation (the how) to be
determined by an independent process. This
makes the semantics (meaning) of declarative
data elements even more critical as these systems
are shared between organizations. This
presentation:
– Provides a background of declarative programming
– Describes why understanding the semantic aspects
of declarative systems is critical to cost-effective
software development
M
D
Metadata Solutions
Note: All opinions stated in this paper are solely those of the author.
2
Presentation Summary (cont)
– Discusses declarative and semantic aspects of
common development systems such as; XHTML,
CSS, XForms, XML transforms, XML Schemas, OWL,
metadata registries, web services, composition,
service-oriented architectures (SOA) and the
enterprise service bus (ESB)
– Discusses how social networking software and Wikis
are used to quickly build consensus on precise
semantics
– Presents ten specific recommendations to lower costs
of agile information systems
M
D
Metadata Solutions
3
Presentation Includes
• Definitions of declarative systems and contrasts with
them with traditional procedural systems and standalone declarative languages
• A critical analysis of semantics in declarative systems
• Case studies using XForms, Wikis and other
collaborative software
• The role of social networking systems, reputation and
trust in the development of semantically precise
declarative frameworks
• Specific recommendations of how organizations can be
more effective by integrating semantics and declarative
systems into their software development processes
M
D
Metadata Solutions
4
Evolution Metaphors
• Specialization of Languages
• Generalization of Languages
M
D
Metadata Solutions
5
Evolution: Specialization
• Darwin’s Galapagos Finches
• Beaks are highly adapted to different food sources
• Finches adapted to specific ecological "niches“ over
millions of years of isolated evolution
• Similar to domain-specific declarative languages
M
D
Metadata Solutions
See Wikipedia "Darwin's Finches"
6
Evolution: Generalization
• Generalization: The Raccoon
– The world has a higher population of raccoons today
due to their ability to quickly adapt to changing urban
environments
– Similar to highly adaptive procedural languages
M
D
Metadata Solutions
7
Computer Science Abstractions
Higher abstractions
Declarative
<xf:input>
<xf:label>
Object
Class MyClass(
Method MyMethod
Structured
Fortran
Assembly
Function(A, B)
FOR I = 1 TO 10 DO
1010001010
M
D
Metadata Solutions
time
8
The Software Development Process
Requirements
Test
(QA Staff)
M
D
Metadata Solutions
(BA)
Declarative Languages
Build
Design
(Programmer)
(Architect)
• Requirements are about “What”
• Design and Build is about “How”
9
A Declarative “System” Is…
• A software development system, tailored to a specific domain
(such as web applications), used to capture precise business
requirements within the context of a problem domain (the
implicit context)
• Declarative systems do not specify how requirements are
implemented to build working systems. Declarative systems
only define the requirements
• Declarative systems document requirements in specialized
vocabularies and can be used to generate entire working
systems including user interfaces, persistence and test data
• Declarative systems specifically omit some assumed
requirements (such as system availability, performance,
reliability, security etc)
A Declarative system is a set of "little languages" with precise
semantics that fit together like a puzzle to solve a problem
M
D
Metadata Solutions
10
Computer Science Definition
Declarative
Languages
Have sub-types
Functional
Languages
Logic
Languages
Constraint
Languages
• Do not confuse a “Declarative
System” with the computer
science language taxonomy
“Declarative Language”
• “Declarative languages" are
used to describe a group of
programming languages and to
contrast them against
imperative languages.
M
D
Metadata Solutions
See Wikipedia “Declarative Programming”
11
Declarative Systems and Context
• Declarative Systems are specialized languages
for capturing requirements within a specific
domain
• Just as the word “play” connotes meaning based
on context* (i.e. theater vs. a playground), a
given vocabulary has the ability to capture
requirements based on the current problem
• The vocabulary for capturing electronic form
requirements (XML Schema) may not be
appropriate for expressing your build process
(Apache Ant)
M
D
Metadata Solutions
* See: http://wordnet.princeton.edu
12
HTML, CSS and SQL
<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Declarative Systems</title>
</head>
<body>
<h1>Introduction to Declarative Systems</h1>
<p class="author">Dan McCreary</p>
<p class="date">April 2007</p>
</body>
</html>
/* global CSS used by all web pages */
body {
font-family: Arial, Helvetica, sans-serif;
font-size: 75%;
margin: 0;
padding: 0;
width: 1000px;
}
h1 {
color: blue;
padding: 0 15px;
}
HTML
CSS
xquery version "1.0";
<html><body>
<h1>Old Expensive Books</h1>
<ul>{
for $book in doc("books.xml")//book
order by $book/title
return
<li>
{$book /title},
{$book}/author},
{$book}/price},
{$book}/pubyear}
</li>
}</ul></body></html>
XQuery
http://www.w3.org/TR/REC-html32#body
• HTML, CSS, XQuery and SQL are
declarative languages using these
definitions
• The semantics or “meaning” of
each tag in the file is determined by
an external organization
• The syntax does not have to be
XML
M
D
Metadata Solutions
-- Old expensive books
SELECT title, author, price, pubyear
FROM books
WHERE
price > 100 AND
pubyear BEFORE ‘1960’
ORDER BY title
SQL
13
XML Schema Sample
• Screen capture of Altova XML Spy
• 30 minutes to learn graphical notation
M
D
Metadata Solutions
See: http://www.altova.com/products/xmlspy/graphical_xml_schema_editor.html
14
Use Case: Electronic Forms
• User fills out a webbased form
• Typical requirements
may include listing of
data to be gathered,
data types, validation
codes and data
repetition patterns
• Examples:
– HTML Forms
– XForms
– InfoPath™
M
D
Metadata Solutions
15
Forms Players
<html>
<head>
<xf:model>
<xf:/model>
<head>
<body>
<xf:input>
</xf:input>
</body>
</html>
XForms
Extension
Mobile Client
Netfront Mobile
Forms Server
My-XForm.xhtml
Rendered
Form
XForms
“Players”
M
D
Metadata Solutions
See Wikipedia “XForms”
16
What Is Declarativeness for A Context?
Schema Drawing Tool
XML Schema File (.xsd)
XForms
xf|input:required
{font-weight: bold;}
xf|input:required .xf-value
{background-color:#fff6af;}
*:required::after
{font-weight:bold;
font-size:1.5em;
content: "*"; color: red; }
XForms CSS tags
M
D
Metadata Solutions
• Efficiency at capturing
the testable business
requirements in a
semantically precise
and concise manner
• Example: Is a field
optional? Will validation
of the data fail if the field
is missing?
• Ability of each data
element to have precise
meaning over time and
within organizations
17
Declarative Spectrum
General Purpose
Narrow Purpose
Less Abstract
More Abstract
Apache Ant
C#
Assembly
Language
C
C++
Java
JavaScript
perl
M
D
Python
Groovy
Ruby
XSL
XQuery
XUL
CSS
HTML
XML Schema
XForms
XQuery Update
XPath
SQL
• For any given context different “languages” have
different levels of “declarativeness”
• General purpose languages are less abstract but can
solve a wide variety of problems
• Declarative languages have a more narrow purpose
and target a specific problem like styling a web page or
selecting data
Metadata Solutions
18
Middle-Tiers Remain Procedural
declarativeness
user
database
Presentation/style
(client tier)
•
•
•
•
M
Business Logic
(middle tier)
Data definition, insert,
selection and update
(persistence tier)
When the interface is consistent, declarative languages flourish
Middle tiers tend to have the most variation
Wikipedia lists over 200 web application frameworks
CMSMatrix.org lists over 700 content management systems
D
Metadata Solutions
See: http://en.wikipedia.org/wiki/List_of_web_application_frameworks and http://www.cmsmatrix.org
19
The Application Development Puzzle
Semantics
Constraints
Presentation
Workflow
Query
Update
Forms
Publish
Build
Transform
M
D
Metadata Solutions
20
Filling In Each Piece
With A Declarative Language
Metadata
Registry
BPEL
XMLSchema
CSS
HTML
XQuery
XQuery Update
XForms
Cocoon
Apache
Ant
XSL
M
D
Metadata Solutions
21
Semantically Precise Vocabularies
Standard
Domain
Atom/RSS
Syndication/Notification
ACORD
Insurance Industry Forms Data
FpML
Financial Products Markup Language
FixML
Banking (Financial Information eXchange)
HL7
Healthcare (now using XML)
MISMO
Mortgage Industry
NIEM
National Information Exchange Model (registry)
OTA
OpenTravel Alliance (Travel industry)
SIF
School Interoperability Framework
GJXDM
US Department of Justice (registry)
XBRL
XML Business Reporting Language
M
D
Metadata Solutions
See: http://en.wikipedia.org/wiki/Category:XML-based_standards
22
Metadata Shopping Tools
Phone
Address
FirstName
• You don’t need to know about
100,000 SKUs to purchase 10
items from a grocery store
• Sub-schema generation tools give
you exactly what you need and
nothing more
M
D
Metadata Solutions
See http://niem.gtri.gatech.edu/iepd-ssgt/SSGT-SearchSubmit.do
23
Criteria for Semantic Precision
• Is there a published standard?
• Are there ISO/IEC 11179 definitions?
– concise, precise, non-circular, distinct
• Are people using it?
– Do a Google search
– > 100,000 and you are safe
– < 10,000 and you should be concerned
Examples: filetype:owl, filetype:xsd
M
D
Metadata Solutions
See “Metadata publishing” Wikipedia
24
If You Use Industry Standards…
You Could Be Almost Done…
• If you use industry standards…
– and these standards publish their documents in XML
Schema format…
– and these standards have been transformed from XML
Schema to XForms…
– and you use native XML databases to store and
XQuery to report on the data…
• …then sample applications have been created
and do not require additional procedural code
– just change the constraints in the XML Schema and
rerun the transforms
M
See: http://www.exist-db.org
D
Metadata Solutions
See also: http://www.alphaworks.ibm.com/tech/purexml
25
Accessibility
• Architecture and
Strategy
(prevent unnecessary
procedural code)
IT
Strategists
Procedural
Programmers
Business Analysts
SMEs and GUI Tools Users
• Extend declarative
vocabularies and
provide web service
“glue”
• Precisely specify
business requirements
• Requires data stewardship
training
M
D
Metadata Solutions
Lower costs by moving routine logic maintenance to lower levels in the pyramid
26
Java Libraries
10,000 class and 100,000+ methods available
…but which ones are relevant to your business problem?
M
D
Metadata Solutions
27
“Less is More”
• XForms 1.1 has
only 21 XML
elements
• Much of the
presentation of
XForms is
deferred to CSS
• Event
management is
deferred to the
XML Events
• XML binding is
deferred to the
XBL standard
M
D
Metadata Solutions
Mies van der Rohe
Reconstruction of the
German Pavilion in Barcelona
See Wikipedia “Minimalism”
28
Learning XForms Vocabulary
1. Bind
2. Case
3. Input
4. Instance
5. Group
6. Help
7. Hint
8. Label
9. Load
10. Output
11. Message
12. Model
13. Repeat
14. Secret
15. Select
16. Select1
17. Switch
18. Submission
19. Submit
20. Textarea
21. Trigger
M
D
Metadata Solutions
Source: W3C XForms Quick Reference http://www.w3.org/MarkUp/Forms/2006/xforms-qr.html
29
Recognizing XForms Structures
1. Bind
2. Case
3. Input
4. Instance
5. Group
6. Help
7. Hint
8. Label
9. Load
10. Output
M
D
Metadata Solutions
11. Message
12. Model
13. Repeat
14. Secret
15. Select
16. Select1
17. Switch
18. Submission
19. Submit
20. Textarea
21. Trigger
Color coding limited vocabularies can increase the speed of pattern matching.
Look for advanced text editors to provide custom element coloring.
30
The New Semantics of "Nutshell"
• The 1.4 release of Java 2
Standard edition increases
the size of the platform by
50%, to 2,757 classes in
135 packages
• 1.5 and 1.6 add additional
classes
992 pages
nutshell: something of small size, amount, or scope
in a nutshell : in a very brief statement
M
D
Metadata Solutions
31
Procedural Programming is Not “Poison”
• It would be a mistake to tell all
your procedural programmers that
the programs they are creating are
fundamentally evil
• The relevant questions are:
- How closely does it fit the problem
domain?
- Can BAs, SMEs and other nonprogrammers maintain the business
rules?
- What are the chances that others will
be able to maintain it in future years?
- How good are the development tools
for your system?
M
D
Metadata Solutions
32
Popular Language Have Better Tools
Limited resource cost curve
$
• Editor
– Syntax coloring
demand
Limited supply curve
Whuffie curve
quantity
Whuffie is a reputation based
currency. Prices drop as
demand increases.
• Debugger
– Set breakpoints
– View internal state variables
• Refactoring tools
– Can recognize reoccurring
patterns and suggest
alternatives
• Performance
– Code profiling
M
D
Metadata Solutions
The higher demand for a good debugger, the better open-source products will become.
33
If You Give a Kid a Hammer…
…the whole world becomes a nail
• People solve problems using
familiar tools
• People develop specific
Cognitive Styles* based on
training and experience
• What are we teaching the next
generation of developers?
M
D
Metadata Solutions
* Source: Shoshana Zuboff: In the Age of the Smart Machine (1988)
34
Use Case: Build Scripts
• Instructions for compiling
source code or
transforming data
• Vocabulary includes terms
such as build, compile,
transform, copy or clean
• Examples:
– Apache Ant
– Apache Maven
– UNIX™ make
M
D
Metadata Solutions
35
Use Case: Data Selection With XQuery
• Ways to specify what data
you want to extract from a
data set
• Typical tasks include
selecting attributes
(columns), filtering,
restricting results and
changing sort order
• Examples
– Structured Query Language
(SQL)
– XQuery (w3c standard)
– FLOWR
M
D
Metadata Solutions
36
Semantics
• The science of meaning
• What you mean when you say “cat”
• How do you associate meaning with
symbols (verbal, physical, textual)
• How do we know if we both mean the
same thing when we use a word?
• What if a word has multiple meanings?
M
D
Metadata Solutions
Reference: WordNet
37
Semantic Triangle
concept
“cat”
referent
symbol
• Symbols can only link to referents through concepts
• You can not link directly from a symbol to a referent
M
D
Metadata Solutions
Wikipedia: Semiotic triangle
38
Communication
“cat”
• Domestic feline
• House pet
• Has fur
• Has whiskers
• Sometimes has fleas
• Chases mice
• Domestic feline
• House pet
• Has fur
• Has whiskers
• Sometimes has fleas
• Chases mice
• Communication involves exchanging symbols that
describe common attributes
• A one-to-one match of attributes that describe a common
symbol match implies a high precision match
M
D
Metadata Solutions
39
Same symbol – different meaning
“customer”
• First name
• Last name
• Home address
• E-mail
• Cell phone number
• Gender
M
D
Metadata Solutions
• Company name
• Home office address
• Branch office address
• CEO name
• Web site
• Industry code
• Little match of common attributes
• Low precision semantic match
40
High and Low Precision
precise standard
precise usage
best
precise standard
vague usage
better
vague standard
vague usage
Not-so-good
• The ideal is to have a precise standard and to use the
data elements exactly as they were intended
M
D
Metadata Solutions
41
Semantic Precision
• Semantic mappings are relative in time and
between groups of people (organization)
• Semantic variability over time
– Something that has precise meaning to you today
may not have the same precise meaning a year from
now
– Our memories are imperfect and change over time
• Semantic variability across organizations
(project and organization)
– A “customer” to one organization may denote a
person but to another organization it may denote a
company
M
D
Metadata Solutions
42
Semantics of an XML Data Element
<code>47</code>
• A developer puts an XML data element in an xml file
• The tag has some meaning and the data within the tag
has some meaning when it was created by the developer
• What is the probability
– That the developer will know the same meaning of the code 47
one year later
– That another project that opens file will understand and be able
to use the meaning of the tag
– Vague standards often trigger vague interpretations of the
meaning of data
M
D
Metadata Solutions
43
Semantic Precision in Space and Time
space: (projects, organizations)
Large
Semantic
Footprint
(long lifetime
systems)
world
enterprise
dept.
team
person
Small Semantic
Footprint
(rapid prototype)
time
M
D
Metadata Solutions
weeks
months
years
10+ years
44
DRY Coding, XSL and MDA
• If developers can’t quickly transform it…
…they will copy it.
• DRY: Don’t Repeat Yourself
• Documentation
– Is always kept up to date?
– Do developers communicate their intent?
– JavaDoc – generated from the source code
• The tendency to copy and paste is just too
common
– Developers must be diligent
– Budgets must be adequate
– Time must be sufficient
• You can promote Model-Driven Architecture
(MDA) by reducing the effort of transformation
from specifications captured in declarative
languages
M
D
Metadata Solutions
45
Use Procedural “Glue”
XForms XML
(REST)
Procedural
“Glue”
XQuery Update
(JDBC)
• XForms uses a REST interface to send XML to a server
• Some XML databases (DB2 v9) sill use JDBC interfaces to insert
XML documents
• Use procedural glue to build custom interfaces between systems
with incompatible interfaces
M
D
Metadata Solutions
46
How Quickly Can We Create New Declarative
Languages?
• Configuration files are really just small
languages
• Configuration files are easy to parse,
validate data using XML tools and build
custom forms to use
• Easy to teach non-programmers to
graphically build XML Schemas to validate
XML files
– Answer – About a week
M
D
Metadata Solutions
47
How Quickly Can We Create Consensus?
• Example: XForms standard
– Work started in 2001
– XForms 2.0 is still a two to three years away
• How can we accelerate this consensus
building process?
What factors impact the rate that species evolve?
What impact does life span have on a species?
How are design ideas exchanged between species?
M
D
Metadata Solutions
48
Solution: Wikis and Collaboration
• How long does it take to build consensus on the
semantics of a new data element?
• How many people might use this declarative
language?
• The larger the stakeholder group, the longer it takes
M
D
Metadata Solutions
49
Relative
Code Base
Parker Projection
100%
M
D
Metadata Solutions
Time
Source: Jason Parker, Minnesota Department of Revenue, November 2006
50
Top 10 Recommendations
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Use semantically-precise declarative systems over procedural systems
when the business problem is in the right context
Aggressively promote data stewardships within business units
Use controlled vocabularies and metadata shopping tools to allow nonprogrammers to select data elements from lists
Separate semantics (leaf elements) from constraints (branches)
Leverage the pattern matching portion of the brain using graphical
development tools and element-based syntax coloring editors
Train programmers on ways to extend declarative vocabularies using
technologies such as XML Binding Language (XBL)
Use procedural code to build interfaces between incompatible systems
Isolate procedural by providing XML/REST service wrappers code behind
semantically precise web services
Don’t expect your procedural programmers to jump on the declarative
bandwagon without some training
Use social networking tools (voting, wikis) to rank and find appropriate
tags and extensions
M
D
Metadata Solutions
51
Unanswered Questions
M
1. How will declarative systems evolve? Will there be a few or many?
2. How quickly will declarative languages evolve? Will they continue to
specialize?
3. Can we deliver entire working systems without ever leaving a single
family of standards? (see XForms/eXist and w3c example)
4. What will be the impact of semantic wikis and micro-formats?
5. Can individual tags stand alone or do they need to be integrated into
larger standards to be accepted?
6. Will metadata shopping tools and graphical constraint tools make the
process of building entire systems accessible to mere mortals?
7. Will declarative systems halt overseas out-sourcing of procedural
software developers?
8. Will there be a large increase in the demand for staff that can extend
declarative systems?
9. Will declarative systems allow applications to run anywhere a “Forms
Players” can run?
10. Will vendors used to locking customers into proprietary APIs be
forced to compete by having more robust and more reliable execution
platforms?
D
Metadata Solutions
52
Selected References
•
Evolution
– Charles Darwin, Origin of the Species
– Richard Dawkins, The Blind Watchmaker
•
Cognitive Styles
– Shoshana Zuboff: In the Age of the Smart Machine (1988)
•
Building Trust
– Succeeding with Objects: Decision Frameworks for Project Management, Adele
Goldberg and Kenneth S. Rubin
•
XML Data Standards
– NIEM Concept of Operations at NIEM.gov
– ISO/IEC 11179 (wikipedia)
•
XForms
– XForms.org, wikipedia
•
Domain Specific Languages
– Niel Ford, Language Oriented Programming
– Martin Flowler
http://www.infoq.com/presentations/domain-specific-languages
M
D
Metadata Solutions
53