Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Topic 3: Finding, Representing & Exploiting Structure Getting Structure: Allow structure specification languages XML? [More structured than text and less structured than databases] If structure is not explicitly specified (or is obfuscated), can we extract it? Wrapper generation/Information Extraction Using Structure: For retrieval: Extend IR techniques to use the additional structure For query processing: (Joins/Aggregations etc) Extend database techniques to use the partial structure For reasoning with structured knowledge Semantic web ideas.. Structure in the context of multiple sources: How to align structure How to support integrated querying on pages/sources (after alignment) Structure An employee record [SQL] A generic web page containing text [English] A movie review [XML] • How will search and querying on these three types of data differ? Slides adapted from Rao (ASU) & Franklin (Berkeley) Structure helps querying • Expressive queries keyword SQL XML • Give me all pages that have key words “Get Rich Quick” • Give me the social security numbers of all the employees who have stayed with the company for more than 5 years, and whose yearly salaries are three standard deviations away from the average salary • Give me all mails from people from ASU written this year, which are relevant to “get rich quick” Slides adapted from Rao (ASU) & Franklin (Berkeley) Adapting old disciplines for Web-age • Information (text) retrieval – Scale of the web – Hyper text/ Link structure – Authority/hub computations • Databases – Multiple databases • Heterogeneous, access limited, partially overlapping – Network (un)reliability • Datamining [Machine Learning/Statistics/Databases] – Learning patterns from large scale data Slides adapted from Rao (ASU) & Franklin (Berkeley) Why do we care about databases? • Three reasons – Deep web is all databases… – We can do better with structured data… – Exposing databases on web changes their clientele.. Slides adapted from Rao (ASU) & Franklin (Berkeley) Deep Web is databases.. • The crawlable web pages are just the tip of a huge ice berg that is deep web – Many web sites have huge backend databases that generate pages dynamically in response to queries • Airline fare databases; News paper classifieds etc. – By some estimates, deep web is 2 orders of magnitude bigger than the shallow (“html page”) web • We need to exploit deep web – Crawl/index deep web – Select databases relevant to a query – Provide information aggregation/integration services over deep web databases • ..and all the big kids are trying to gobble up anyone who is even going through the motions of doing these.. • …which leads to several DB challenges not addressed in traditional DBs – – – – Wrapper generation Schema mapping [Structure alignment] (automated) form filling Query optimization • Learning source profiles Slides adapted from Rao (ASU) & Franklin (Berkeley) Databases offer lessons on exploiting structure • We argued that structure (and semantics) help querying – If there is structure (as in databases) we can exploit it • Databases is an existing technology for exploiting some forms of structure – SQL may not look like much, but it is more expressive than keyword queries! – If not, we can extract structure and then exploit it • Challenges – Techniques for extracting information (NLP-lite) – Languages for representing/handling “Semi-structured” data – Standards for supporting/exploiting semantic tagging Slides adapted from Rao (ASU) & Franklin (Berkeley) Before we play havoc with databases, let’s quickly review the traditional art of db management so we know all that needs to change Databases !!!??? you may have used Slides adapted from Rao (ASU) & Franklin (Berkeley) What Is a Database System? • Database: a very large, integrated collection of data. • Models a real-world enterprise – Entities (e.g., teams, games) – Relationships (e.g., The Patriots are playing in The Superbowl) – More recently, also includes active components , often called “business logic”. (e.g., the BCS ranking system) • A Database Management System (DBMS) is a software system designed to store, manage, and facilitate access to databases. Slides adapted from Rao (ASU) & Franklin (Berkeley) Functionality of a DBMS • Data Dictionary Management • Storage management – Data storage Definition Language (DDL) • High level query and data manipulation language – SQL/XQuery etc. – May tell us what we are missing in text-based search • Efficient query processing – May change in the internet scenario • Transaction processing • Resiliency: recovery from crashes, • Different views of the data, security – May be useful to model a collection of databases together • Interface with programming languages Slides adapted from Rao (ASU) & Franklin (Berkeley) Traditional Database Architecture Database (relational) Services Source Trust Ontologies; Source/Service Descriptions Probing Queries Webpages Structured data od M ity til /U ce en er Executor Answers Needs to handle Source/network Interruptions, Runtime uncertainity, replanning cs tisti g Sta atin Upd ing nn pla ts Re ques Re e Qu Slides adapted from Rao (ASU) & Franklin (Berkeley) lls el ry Sensors (streaming Data) Ca Needs to handle: Multiple objectives, Service composition, Source quality & overlap So ur ce Source Fusion/ Query Planning ef Answer (relation) Database Manager (DBMS) -Storage mgmt -Query processing -View management -(Transaction processing) Pr Query (SQL) Monitor Building an Application with a Database System • Requirements modeling (conceptual, pictures) – Decide what entities should be part of the application and how they should be linked. • Schema design and implementation – Decide on a set of tables, attributes. – Define the tables in the database system. – Populate database (insert tuples). • Write application programs using the DBMS – Now much easier, with data management API Slides adapted from Rao (ASU) & Franklin (Berkeley) Conceptual Modeling name category name ssn Takes Course Student quarter Advises Teaches Professor address name Slides adapted from Rao (ASU) & Franklin (Berkeley) field Data Models • A data model is a collection of concepts for describing data. • A schema is a description of a particular collection of data, using a given data model. • The relational model of data is the most widely used model today. – Main concept: relation, basically a table with rows and columns. – Every relation has a schema, which describes the columns, or fields. Slides adapted from Rao (ASU) & Franklin (Berkeley) Levels of Abstraction • Views describe how users see the data. • Conceptual schema defines logical structure • Physical schema describes the files and indexes used. View 1 View 2 View 3 Conceptual Schema Physical Schema Slides adapted from Rao (ASU) & Franklin (Berkeley) DB Example: University Database • Conceptual schema: – Students(sid: string, name: string, login: string, age: integer, gpa:real) – Courses(cid: string, cname:string, View 1 credits:integer) • External Schema (View): – Course_info(cid:string,enrollment:in teger) • Physical schema: – Relations stored as unordered files. – Index on first column of Students. View 2 View 3 Conceptual Schema Physical Schema DB If fiveadapted peoplefrom are Rao asked to come up with a schema for the data, Slides (ASU) & Franklin (Berkeley) what are the odds that they will come up with the same schema? Data Independence • Applications insulated from how data is structured and stored. • Logical data independence: Protection from changes in logical structure of data. View 1 View 2 View 3 Conceptual Schema Physical Schema • Physical data independence: Protection from changes in physical structure of data. • Q: Why are these particularly important for DBMS? Slides adapted from Rao (ASU) & Franklin (Berkeley) DB Schema Design & Implementation • Table Students Student Course Quarter Charles CS 444 Fall, 1997 Dan CS 142 … … Winter, 1998 … • Separates the logical view from the physical view of the data. Slides adapted from Rao (ASU) & Franklin (Berkeley) Terminology Attribute names tuples Students Student Course Quarter Charles CS 444 Fall, 1997 Dan CS 142 … … Winter, 1998 … (Arity=3) Slides adapted from Rao (ASU) & Franklin (Berkeley) Querying a Database • Find all the students taking CSE594 in Q1, 2004 • S(tructured) Q(uery) L(anguage) select E.name from Enroll E where E.course=CS490i and E.quarter=“Winter, 2000” • Query processor figures out how to answer the query efficiently. Slides adapted from Rao (ASU) & Franklin (Berkeley) Defining Views (Virtual) Views are “macro” relations defined in terms of base relations (they may or may not be physically stored) They are used mostly in order to simplify complex queries and to define conceptually different views of the database to different classes of users. View: purchases of telephony products: CREATE VIEW telephony-purchases AS SELECT product, buyer, seller, store FROM Purchase, Product WHERE Purchase.product = Product.name AND Product.category = “telephony” Slides adapted from Rao (ASU) & Franklin (Berkeley) A Different View CREATE VIEW Seattle-view AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = “Seattle” AND Person.name = Purchase.buyer We can later use the views: SELECT name, store FROM Seattle-view, Product WHERE Seattle-view.product = Product.name AND Product.category = “shoes” What’s really happening when we query a view?? Slides adapted from Rao (ASU) & Franklin (Berkeley) Updating Views How can I insert a tuple into a table that doesn’t exist? CREATE VIEW bon-purchase AS SELECT store, seller, product FROM Purchase WHERE store = “The Bon Marche” If we make the following insertion: INSERT INTO bon-purchase VALUES (“the Bon Marche”, Joe, “Denby Mug”) We can simply add a tuple (“the Bon Marche”, Joe, NULL, “Denby Mug”) to relation Purchase. Slides adapted from Rao (ASU) & Franklin (Berkeley) Non-Updatable Views Given Purchase (buyer, seller, store, product) Person( name, phone-num, city) CREATE VIEW Seattle-view AS SELECT seller, product, store FROM Person, Purchase WHERE Person.city = “Seattle” AND Person.name = Purchase.buyer How can we add the following tuple to the view? (Joe, “Shoe Model 12345”, “Nine West”) Slides adapted from Rao (ASU) & Franklin (Berkeley) Materialized Views • Views whose corresponding queries have been executed and the data is stored in a separate database – Uses: Caching • Issues – Using views in answering queries • Normally, the views are available in addition to database – (so, views are local caches) • In information integration, views may be the only things we have access to. – An internet source that specializes in woody allen movies can be seen as a view on a database of all movies. Except, there is no database out there which contains all movies.. – Maintaining consistency of materialized views Slides adapted from Rao (ASU) & Franklin (Berkeley) Query Optimization Goal: Declarative SQL query SELECT S.buyer FROM Purchase P, Person Q WHERE P.buyer=Q.name AND Q.city=‘seattle’ AND Q.phone > ‘5430000’ Imperative query execution plan: buyer City=‘seattle’ phone>’5430000’ Buyer=name Inputs: • the query • statistics about the data (indexes, cardinalities, selectivity factors) • available memory Purchase (Table scan) (Simple Nested Loops) Person (Index scan) Ideally: Want to find best plan. Practically: Avoid worst plans! Slides adapted from Rao (ASU) & Franklin (Berkeley) Web brings unwashed masses, unreliable medium as well as dirty data to databases.. • Web accessibility changes the user/data/medium profile significantly – from SQL gurus supporting financial data on dedicated DBMS to “2.1 keyword query” instant gratification seekers working with dirty/inconsistent data over unreliable web. • Challenges – – – – How does one support keyword queries in databases? How does one support imprecise queries in databases? How do we handle incompleteness/inconsistency in databases? Does it make sense to focus on total response time minimization • As against a multi-objective cost/benefit optimization? The DB community has embraced these challenges --see Lowell Report Slides adapted from Rao (ASU) & Franklin (Berkeley) Specifying Structure: The XML Standard 11/18 Specifying Structured Text/Data: XML • XML is the confluence of several factors: – The Web needed a more declarative format for data, trying to describe the meaning of the data – Documents needed a mechanism for extended tags to mark structure – Database people needed a more flexible interchange format • Original expectation: – The whole web would go to XML instead of HTML • Today’s reality: – Not so… But XML is used all over “under the covers” 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) TEXT More Structure XML Less Structure Structured (relational) Data Differing Expectations Based on which Side you came70from An XML Document Example Start Tag <imdb> <show year=“1993”> <title>Fugitive, The</title> <review> <suntimes> <reviewer>Roger Ebert</reviewer> gives <rating>two thumbs up</rating>! A fun action movie, Harrison Ford at his best. </suntimes> --can be nested </review> <review> <nyt>The standard &hollywood; summer movie strikes back.</nyt> </review> <box_office>183,752,965</box_office> </show> <show year=“1994”> <title>X Files,The</title> <seasons>4</seasons> </show> Slides adapted from Rao (ASU) & </imdb> Mixed Content Element End Tag Attribute 5/5/2017 Franklin (Berkeley) 71 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 72 HTML vs. XML <h1> Bibliography </h1> <p> <i> Foundations of Databases </i> Abiteboul, Hull, Vianu <br> Addison Wesley, 1995 <p> <i> Data on the Web </i> Abiteoul, Buneman, Suciu <br> Morgan Kaufmann, 1999 5/5/2017 <bibliography> <book> <title> Foundations… </title> <author> Abiteboul </author> <author> Hull </author> <author> Vianu </author> <publisher> Addison Wesley </publisher> <year> 1995 </year> </book> … </bibliography> Slides adapted from Rao (ASU) & Franklin (Berkeley) 73 <h1> Bibliography </h1> <p> <i> Foundations of Databases </i> Abiteboul, Hull, Vianu <br> Addison Wesley, 1995 <p> <i> Data on the Web </i> Abiteoul, Buneman, Suciu <br> Morgan Kaufmann, 1999 HTML describes presentation 5/5/2017 <bibliography> <book> <title> Foundations… </title> <author> Abiteboul </author> <author> Hull </author> <author> Vianu </author> <publisher> Addison Wesley </publisher> <year> 1995 </year> </book> … </bibliography> XSL (stylesheets) XML describes can be used to Slides adapted from Rao (ASU) & specify the conversion Franklin (Berkeley) content 74 XML Terminology • • • • • • tags: book, title, author, … start tag: <book>, end tag: </book> elements: <book>…<book>,<author>…</author> elements are nested empty element: <red></red> abbrv. <red/> an XML document: single root element well formed XML document: if it has matching tags 5/5/2017 Slides adapted from Rao (ASU) & 75 Franklin (Berkeley) XML & Order • If you see an XML file as a text file with tags, then order should matter • If you see an XML file as a self-describing version of (relational) data, then order shouldn’t matter • Which should be the default? 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 77 Object identifiers More XML: Oids and References <person id=“o555”> <name> Jane </name> </person> <person id=“o456”> <name> Mary </name> <children idref=“o123 o555”/> </person> <person id=“o123” mother=“o456”><name>John</name> </person> oids and references in XML are just syntax Slides adapted from Rao (ASU) & 79 5/5/2017 Franklin (Berkeley) XML & Meaning Jim Hendler XML machine accessible meaning This is what a web-page in natural language looks like for a machine (Unless it is in Beijing.. ) 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 82 XML machine accessible meaning Jim Hendler XML allows “meaningful tags” to be added to parts of the text < name > < education> < CV > < work> < private > 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 83 XML machine accessible meaning Jim Hendler But to your machine, the tags look like this….(assuming it is not in Athens) name > < name <education> < education> < CV CV > <work> < work> <private> < private > 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 84 XML machine accessible meaning Jim Hendler Schemas help…. < CV > private 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) …by relating common terms between documents 85 But other people use other schemas Jim Hendler Someone else has one like this…. <name> name > <<educ> education> << CV CV >> <> > < work <<> private > 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 86 But other people use other schemas Jim Hendler < CV > private 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) …which don’t fit in 87 XML & Meaning: Summary • XML is a purely syntactic standard – Saying that something is in XML format is like saying something is in List or Table format • It is NOT like saying that something in English/C++ etc (all of which have specific semantics) • Tags in XML do not up front have any “meaning” – Tags can be overloaded with specific meaning through prior agreement or standardization • Such agreements/standardization are possible for specific sub-tasks (e.g. HTML for rendering) or specific sub-communities (e.g. ebXML etc—see next slide) – Tags’ meaning can be expressed by relating them to other tags • This is the usual knowledge representation way (meaning comes from inter-predicate relations). Semantic Web pushes this view. 5/5/2017 – You can also learn the relations through context/practice/usage etc. This is the sort of view taken by (semi-automated) schema-mapping techniques Slides adapted from Rao (ASU) & 88 Franklin (Berkeley) XML Dialect “pot pourri” Extensible Financial Reporting Markup Language (XFRML), eXtensible Business Reporting Language (XBRL), MusicXML, Spacecraft Markup Language (SML), Bank Internet Payment System (BIPS), Bioinformatic Sequence Markup Language (BSML), Biopolymer Markup Language (BIOML), Open Catalog Format (OCF), Chemical Markup Language (CML), Electronic Business XML Initiative (ebXML), Open Trading Protocol (OTP), FinXML, Financial Information eXchange protocol (FIX), RecipeML, CVML, XML Bookmark Exchange Language (XBEL), Scalable Vector Graphics (SVG), NewsML, DocBook, Slides adapted from Rao (ASU) & Franklin (Berkeley) . . . Real Estate Listing Markup Language (RELML), 5/5/2017 Examples of communities that Standardized their tags… 89 Who puts everything into XML? • To a certain extent, this a vaccuous question, once we realize that XML is just a syntactic standard – You can put things into XML by just putting <body> tag (or any tag) at the beginning and end of the file • XML is not meant to be an imposition but rather a facilitator – XML facilitates marking up structure if someone wants to do this. That someone can be: • creator of the page • secondary user who wants to tag the page • An extraction program that wants to remember the structure it extracted by tagging the page – The markup tags may or may not have any specific meaning based on prior agreements/standardization 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 90 Why are IR folks excited about XML? • XML files are text files with structure – Structure easily identifiable (the DOM structure) • We can improve Precision/Recall by taking structure into account.. – We already did a bit—e.g. higher weight to words occuring in the header tags.. 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 91 Why are Database folks excited about XML? • XML is just a syntax for (selfdescribing) data • This is still exciting because – No standard syntax for relational data – With XML, we can • Translate any legacy data to XML • Can exchange data in XML format – Ship over the web, input to any application • Talk about querying on seim-structured data 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 92 XML viewed from a Database Point of View XML vs. Relational Data • XML is meant as a language that supports both Text and Structured Data – Conflicting demands... • XML supports semi-structured data – In essence, the schema can be union of multiple schemas • Easy to represent books with or without prices, books with any number of authors etc. • XML supports free mixing of text and data – using the #PCDATA type • XML is ordered (while relational data is unordered) 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) TEXT More Structure XML Less Structure Structured (relational) Data 94 XML Data Model (DOM) imdb show review title @year “1993” “Fugitive, The” suntimes review nyt …… reviewer rating “Roger Ebert” “gives” “two...” Check http://www.w3.org/XML/ for more details 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 95 DTDs <!DOCTYPE paper [ <!ELEMENT paper (section*)> <!ELEMENT section ((title,section*) | text)> <!ELEMENT title (#PCDATA)> <!ELEMENT text (#PCDATA)> ]> Semistructured <paper> <section> <text> </text> </section> <section> <title> </title> <section> … </section> <section> … </section> </section> </paper> 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 96 XML Schema • • • • • Supersedes DTD (and has XML syntax) unifies previous schema proposals generalizes DTDs uses XML syntax two documents: structure and datatypes – http://www.w3.org/TR/xmlschema-1 – http://www.w3.org/TR/xmlschema-2 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 97 XML Schema 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 98 http://support.x-hive.com/xquery/index.html You will be asked to play with it in homework 3 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 101 FLoWeR Expressions Xquery queries are made up of FLWR expressions that work on “paths” • For binds variables to nodes • Let computes aggregates • Where applies a formula to find matching elements • Return constructs the output elements Path expressions are of the form: element//element/element[attrib=value] 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 102 Comparison to SQL • Look at the use case description on Xquery manual • Supports all (?) SQL style queries (with different syntax of course) [default queries in the demo] • Has support for – “construction”—outputting the answers in arbitrary XML formats (use case “XMP” ) – “path expressions” --- navigating the XML tree (use case “seq”) – Simple text queries [use case “text”] – Allows queries on “Tag” elements • Removes the “data/meta-data” barrier in queries – For each book that has at least one author, list the title and first two authors, and an empty "et-al" element if the book has additional authors. [XMP use case 6] Make-up Class: Wed 26th 10:30AM—Room TBD (probably 210) 11/20 XQuery; IR-style search on XML; Semantic Web standards DTD for http://www.bn.com/bib.xml <!ELEMENT bib (book* )> <!ELEMENT book (title, (author+ | editor+ ), publisher, price )> <!ATTLIST book year CDATA #REQUIRED > <!ELEMENT author (last, first )> <!ELEMENT editor (last, first, affiliation )> <!ELEMENT title (#PCDATA )> <!ELEMENT last (#PCDATA )> <!ELEMENT first (#PCDATA )> <!ELEMENT affiliation (#PCDATA )> <!ELEMENT publisher (#PCDATA )> <!ELEMENT price (#PCDATA )> 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 105 Example Query Query Result <bib> { for $b in /bib/book where $b/publisher = "AddisonWesley" and $b/@year > 1991 return <book year={ $b/@year }> { $b/title } </book> } </bib> <bib> <book year="1994"> <title>TCP/IP Illustrated</title> </book> <book year="1992"> <title>Advanced Programming in the Unix environment</title> </book> </bib> “For all books after 1991, return with Year changed from a tag to an attribute” 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 106 Example Query (2) • Return the books that cost more at amazon than fatbrain Let $amazon := document(http://www.amazon.com/books.xml), Let $fatbrain := document(http://www.fatbrain.com/books.xml) For $am in $amazon/books/book, $fat in $fatbrain/books/book Join Where $am/isbn = $fat/isbn and $am/price > $fat/price Return <book>{ $am/title, $am/price, $fat/price }<book> 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 107 XML frenzy in the DB Community • Now that XML is there, what can we do with it? – Convert all databases from Relational to XML? • Or provide XML views of relational databases? – Develop theory of native XML databases? • Or assume that XML data will be stored in relational databases.. – Issues: What sort of storage mechanisms? What sort of indices? 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 108 RDBMS On the internet, nobody needs to know that you are a dog XML middleware for Databases Xquery • XML adapters (middle-ware) received significant attention in DB community SQL XML Relations – SilkRoute (AT&T) – Xperanto (IBM) • Issues: – Need to convert relational data into XML • Tagging (easy) – Need to convert Xquery queries into equivalent SQL queries • Trickier as Xquery supports schema querying 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 109 IR Style Querying of XML Documents 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 110 From Manning et al IR Text An XML document is represented as a vector in the space of Lexical Trees Query is an extended lexical tree Similarity between Query & Lexical tree defined as follows: Within the document, you return the snippet that is closest.. Note that we are increasing the size of adapted the index (lexical trees&rather than just words), to exploit 5/5/2017 Slides from Rao (ASU) 111 Structure. This is normal (i.e., index becomes larger when structure is present) Franklin (Berkeley) Semantic Web Standards RDF/RDF-Schema/OWL Syntax vs. Semantics • Syntax provides the grammar for a language (all you can do is to see whether a sentence is grammatically correct and do “parts of speech” tagging – XML • Semantics provides the set of worlds where a particular sentence (or a set of sentences) hold – Many formal languages have welldefined semantics (Propositional logic; first order logic etc.) – Semantic Web involves providing an XML syntax for representing “description logics”—a fragment of First order logic • Has two parts: Base facts are represented by RDF standard • Background Knowledge (axioms etc.)are represented by RDFSchema (which is superseded now by OWL) 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) 113 What we want is a standard for representing knowledge on the web.. A standard technique for KR is Logic So how about we find a way of encoding Logical statements in XML? A logical theory consists of – – RDF is a standard for writing (binary predicate) base-facts – Base facts Background theory E.g. parent(Tom,Mary) RDF-Schema is a standard for writing background theory.. – E.g. Forallx,y Parent(x,y)=>Loves(x,y) 116 Recall that the complexity of inference depends on the form of background theory (e.g. semi-decidable for general FOPC and polynomial for Horn clause. It is also tractable for “description logics” where all the background knowledge is of the form class, sub-class, instance. This is what RDF-Schema tries to capture) RQL is (an emerging?) standard for querying RDF/RDF-S databases 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) Basic Ideas of RDF Basic building block: object-attribute-value triple – – RDF has been given a syntax in XML – – 117 It is called a statement Sentence about Billington is such a statement This syntax inherits the benefits of XML Other syntactic representations of RDF possible 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) The RDF Data Model • Statements are <subject, predicate, object> triples: Ia n hasColleague Ul i • Can be represented using XML serialisation, e.g.: <Ian,hasColleague,Uli> • Statements describe properties of resources • A resource is a URI representing a (class of) object(s): – – – – – a document, a picture, a paragraph on the Web; http://www.cs.man.ac.uk/index.html a book in the library, a real person (?) isbn://5031-4444-3333 … • Properties themselves are also resources (URIs) 05/05/2017 120 URIs • URI = Uniform Resource Identifier • "The generic set of all names/addresses that are short strings that refer to resources“ • URIs may or may not be dereferencable – URLs (Uniform Resource Locators) are a particular type of URI, used for resources that can be accessed on the WWW (e.g., web pages) • In RDF, URIs typically look like “normal” URLs, often with fragment identifiers to point at specific parts of a document: – http://www.somedomain.com/some/path/to/file#fragmentID 05/05/2017 121 RDF Syntax • • • • RDF has an XML syntax that has a specific meaning: Every Description element describes a resource Every attribute or nested element inside a Description is a property of that Resource with an associated object resource Resources are referred to using URIs <Description about="some.uri/person/ian_horrocks"> <hasColleague resource="some.uri/person/uli_sattler"/> </Description> <Description about="some.uri/person/uli_sattler"> <hasHomePage>http://www.cs.mam.ac.uk/~sattler</hasHomePage> </Description> <Description about="some.uri/person/carole_goble"> <hasColleague resource="some.uri/person/uli_sattler"/> </Description> 05/05/2017 122 Linking Statements • • The subject of one statement can be the object of another Such collections of statements form a directed, labeled graph hasColleague Ian Uli hasColleague Carole • • hasHomePage http://www.cs.mam.ac.uk/~sattler Note that the object of a triple can also be a “literal” (a string) Note also that RDF triples don’t by themselves give meaning – You know that (1) Ian and Carol are most likely colleagues (barring multiple jobs for Uli (2) (Uli hasCollegue Ian) holds (“colleagueness” – unlike “love” is symmetric). But DOES YOUR PROGRAM KNOW THIS? 05/05/2017 123 A Critical View of RDF: Binary Predicates RDF uses only binary properties – – Example: referee(X,Y,Z) – 124 This is a restriction because often we use predicates with more than 2 arguments But binary predicates can simulate these X is the referee in a chess game between players Y and Z 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) A Critical View of RDF: Binary Predicates (2) We introduce: – – 125 a new auxiliary resource chessGame the binary predicates ref, player1, and player2 We can represent referee(X,Y,Z) as: 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) A Critical View of RDF: Properties Properties are special kinds of resources – – 126 Properties can be used as the object in an object-attribute-value triple (statement) They are defined independent of resources This possibility offers flexibility But it is unusual for modelling languages and OO programming languages It can be confusing for modellers 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) A Critical View of RDF: Reification 127 The reification mechanism is quite powerful It appears misplaced in a simple language like RDF Making statements about statements introduces a level of complexity that is not necessary for a basic layer of the Semantic Web Instead, it would have appeared more natural to include it in more powerful layers, which provide richer representational capabilities 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) A Critical View of RDF: Summary RDF has its idiosyncrasies and is not an optimal modeling language but It is already a de facto standard It has sufficient expressive power – 128 At least as for more layers to build on top Using RDF offers the benefit that information maps unambiguously to a model 5/5/2017 Slides adapted from Rao (ASU) & Franklin (Berkeley) RDF Schema (RDFS) • RDF gives a formalism for meta data annotation, and a way to write it down in XML, but it does not give any special meaning to vocabulary such as subClassOf or type – Interpretation is an arbitrary binary relation – I.e., <Person,subClassOf,Animal> has no special meaning • RDF Schema defines “schema vocabulary” that supports definition of ontologies – gives “extra meaning” to particular RDF predicates and resources (such as subClasOf) – this “extra meaning”, or semantics, specifies how a term should be interpreted 05/05/2017 NOTICE THAT RDF-SCHEMA is NOT to RDF WHAT XML-Schema is to XML 129 “Background Theory” RDF Schema is really RDF background knowledge! “Instances” 05/05/2017 130 RDF/RDFS vs. General Knowledge Rep & Reasoning • • • • We noted that RDF can be seen as “base level facts” and RDFS can be seen as “background theory/facts/rules At this level, inference with RDF/RDFS seems to be just a special case of Knowledge Representation Reasoning This is good (CSE471 Ahoy!) and bad (reasoning over most nontrivial logics is NP-hard or much much worse). RDF/RDFS can be seen as an attempt to limit the complexity of reasoning by limiting the expressiveness of what can be expressed – RDF/RDFS together can be seen as capturing a certain tractable subset of First Order Logic – ..already there is trouble in paradise with people complaining that the expressiveness is not enough • Enter OWL, which attempts to provide expressiveness equivalent to “description logics” (a sort of inheritance reasoning in Firstorder logic) • But what about uncertain knowledge? (e.g. first order bayes nets?)… 05/05/2017 131 It is clear that the complexity of query answering in logical theories depends on the nature of the theory. Since RDF is just base facts, we are particularly interested in what is expressible in RDF-Schema – RDF-Schema turns out to be closest to a fragment/variant of First order logic called “description logic” – 132 Where most of the knowledge is in terms of class/sub-class relationships Turns out that RDF-Schema is not even as expressive as description logic; so now there is a “more expressive” standard called OWL 5/5/2017 But, does it make sense to limit expressiveness of what can be said a priori? An alternative is to let everything be expressed (e.g. at First order logic level), but only support some of the queries (e.g. go with sound but incomplete inference procedures) An argument can be made that this alternative is more closer to the WEB philosophy—where we already let people write anything they want in full natural language, but support limited forms of retrieval.. Slides adapted from Rao (ASU) & Franklin (Berkeley) Added based on the discussion in the class Expressiveness issues in RDF-Schema Intended Use of Semantic Web? • Pages should be annotated with RDF triples, with links to RDF-S (our OWL) background ontology. • E.g. See Jim Hendler’s page… 05/05/2017 140 Who will annotate the data? • Semantic web works if the users annotate their pages using some existing ontology (or their own ontology, but with mapping to other ontologies) – • But users typically do not conform to standards.. • and are not patient enough for delayed gratification… Two Solutions – – – 1. Intercede in the way pages are created (act as if you are helping them write web-pages) • What if we change the MS Frontpage/Claris Homepage so that they (slyly) add annotations? • E.g. The Mangrove project at U. Wash. – Help user in tagging their data (allow graphical editing) – Provide instant gratification by running services that use the tags. 2. Collaborative tagging! • “Folksonomies” (look at Wikipedia article) – FLICKR, Technorati, deli.cio.us etc • CBIOC, ESP game etc. – Need to incentivize users to do the annotations.. 3. Automated information extraction (next topic) 05/05/2017 141 Folksonomies—The good • Bottom-up approach to taxonomies/ontologies – [In systems like] Furl, Flickr and Del.icio.us... people classify their pictures/bookmarks/web pages with tags (e.g. wedding), and then the most popular tags float to the top (e.g. Flickr's tags or Del.icio.us on the right).... – [F]olksonomies can work well for certain kinds of information because they offer a small reward for using one of the popular categories (such as your photo appearing on a popular page). People who enjoy the social aspects of the system will gravitate to popular categories while still having the freedom to keep their own lists of tags. 05/05/2017 142 Works best when Many people Tag the same Info… 05/05/2017 143 Folksonomies… the bad • On the other hand, not hard to see a few reasons why a folksonomy would be less than ideal in a lot of cases: – None of the current implementations have synonym control (e.g. "selfportrait" and "me" are distinct Flickr tags, as are "mac" and "macintosh" on Del.icio.us). – Also, there's a certain lack of precision involved in using simple one-word tags--like which Lance are we talking about? – And, of course, there's no heirarchy and the content types (bookmarks, photos) are fairly simple. • For indexing and library people, folksonomies are about as appealing as Wikipedia is to encyclopedia editors. – But.. there's some interesting stuff happening around them. 05/05/2017 144 Mass Collaboration (& Mice running the Earth) • The quality of the tags generated through folksonomies is notoriously hard to control – So, design mechanisms that ensure correctness of tags.. • ESP game makes it fun to • CBIOC and Google Co-op restrict annotation previleges to trusted users.. • It is hard to get people to tag things in which they don’t have personal interest.. – Find incentive structures.. • ESP makes it a “game” with points • CBIOC and Google Co-op try to promise delayed gratification in terms of improved search later.. 05/05/2017 145