* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Introduction Using SQL from Programs - auf Matthias
Survey
Document related concepts
Entity–attribute–value model wikipedia , lookup
Tandem Computers wikipedia , lookup
Relational algebra wikipedia , lookup
Serializability wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Oracle Database wikipedia , lookup
Microsoft Access wikipedia , lookup
Concurrency control wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Clusterpoint wikipedia , lookup
Database model wikipedia , lookup
Versant Object Database wikipedia , lookup
Relational model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Transcript
Using SQL from Programs 9 Embedding SQL into Programming languages 9.1 9.2 Introduction: using SQL from programs Embedded SQL Static and dynamic embedding Cursors ESQL / C Positioned Update 9.3 SQL and Java JDBC SQLJ 9.4 9.5 OR mapping and components Transactions in application programs Definition Isolation levels Lit.: Kemper / Eickler: chap. 4.19-4.23; Melton: chap. 12,13,17-19, Widom, Ullman, Garcia-Molina: chapt.8 Introduction SQL is a data sublanguage Needs a host language Control structures User interface: output formatting, forms Transactions: more than one DB interaction as a unit of work Issues Language mismatch ("impedance mismatch") • Set oriented operations versus manipulation of individuals -> Objects vs. Relations • How to interconnect program variables and e.g attributes in SQL statements? • Should an SQL-statement as part of a program be compiled, when? © HS-2009 9-Embedded-2 Christian Ullenboom Java ist auch eine Insel, Kap. 20, Galileo Comp. Threeexample) Three-tier architecture (example) GUI client Web browser Middleware layer SQL from Programs: Programs: Introduction Web browser DB client Middle tier Web Server DB Application DB Application File System DB-Server DB-Server © HS-2009 DB-Server 9-Embedded-3 Using SQL from Programs: Programs: Call interface Call level interface Language and DBS specific library of procedures to access the DB Example: MySQL C API • Communication buffer for transferring commands and results • API data types like MYSQL handle for db connections MYSQL_RES structure which represents result set • API functions mysql_real_query() mysql_real_query (MYSQL *mysql, const char * query, unsigned int length) query of length of character string in buffer and many more.... Standard : Open Database Connection (ODBC) Predecessor of Java Database Connection (JDBC), see below © HS-2009 9-Embedded-6 Overview of language / DB integration concepts "Fourth Generation Languages" Module Language --> PSM ( ~ PL/SQL, PLpgSQL) • Standardized in SQL-99 9 Interface of standard programming languages • Call level interface, proprietary library routines, API Standardized: SQL CLI Open Database connection (ODBC), • Embedded C / Java / .. Standardized language extensions • Standardized API Java DBC Object-Relational Mapping • JDO, Hibernate Component architectures: hiding the details of DB interaction, Enterprise Java Beans (EJB) © HS-2009 9-Embedded-4 SQL Call level interface (SQL/CLI) Standardized Interface to C / C++ defined by X/OPEN and SQL Access group • Main advantages DBS-independent Application development independent from DBS (as opposed to Embedded SQL precompiler approach, see below) Easy to connect to multiple DB • Microsoft implementation ODBC (= Open Database Connectivity) de facto standard, available not only for MS products © HS-2009 9-Embedded-7 1 9.2 Embedded SQL Embedded SQL (ESQL): what needed? needed? Embedded SQL – the most important(?) approach Concepts • Program consists of "native" and SQL-like statements • Precompiler compiles it to native code, includes calls to DBS resources • Employs call level interface in most implementations • Most popular: Embedded C (Oracle: PRO*C) • Well defined type mapping (for different languages) ESQL Precompiler Native Language code Compiler Object code • SQLJ = Embedded Java © HS-2009 Linker Excecutable • Syntax for embedded SQLJ statements #sql [[<context>]] { <SQL-Anweisung> } • Binding to host language variables #sql {SELECT m# FROM M syntax, WHERE titel = :titleString};}... hypothetical like SQLJ #sql {FETCH ...INTO :var1} • Exception handling (WHENEVER condition action) SQLSTATE, SQLCODE (deprecated) Library 9-Embedded-9 © HS-2009 9-Embedded-10 ESQL 9.2.2 ESQL Cursors C / Java embedding Cursor concept ESQL/C EXEC SQL UPDATE staff SET job = 'Clerk' WHERE job = 'Mgr'; if ( SQLCODE < 0 printf( "Update Error: ... ); How to process a result set one tuple after the other? CURSOR: name of an SQL statement and a handle for processing the result set record by record Cursor is defined, opened at runtime (= SQL-statement is excecuted) and used for FETCHing single result records DECLARE c .. SQLJ OPEN c FETCH c try { #sql { UPDATE staff SET job = 'Clerk' WHERE job = 'Mgr' }; } catch (SQLException e) { println( "Update Error: SQLCODE = " + ... ); © HS-2009 9-Embedded-11 ESQL Cursors Explicit cursors: Declared and named by the programmer Sometimes implicit cursors for individual SQL statements are used in 4GL © HS-2009 ESQL Cursor curs © HS-2009 SMITH To be or .. 7566 JONES Metropolis 7788 SCOTT Forest Gump 7876 ADAMS Forest Gump 7902 FORD Star Wars I FETCH , ::nameVar, nameVar, titleVar; ; :x nameVar, :titleVar ::titleVar; FETCH curs curs INTO INTO :x, :x, Cursor scrolling (Declare c SCROLL cursor.. in SQL 92): FETCH [NEXT | PRIOR | FIRST | LAST | [ABSOLUTE | RELATIVE expression] ] FROM cursor INTO target-variables = FETCH , ::nameVar, nameVar, titleVar; ; :x nameVar, :titleVar ::titleVar; INTO :x, :x, FETCH curs curs PRIOR PRIOR INTO FETCH 1 INTO , ::nameVar, nameVar, titleVar; ; :x nameVar, :titleVar ::titleVar; FETCH curs curs RELATIVE RELATIVE ––1 INTO :x, :x, Current row 9-Embedded-14 Cursors Fetch assigns a name to an SQL statement. Cursor / SQL statement do not bind the result attributes to variables allows to traverse the result set (the "active set" ) row by row 7369 DBS Cursor concept used in most language embeddings of SQL (ESQL-C, PL/SQL, JDBC and more) Buffers for application program cursors DBS may determine result set in a lazy or eager way 9-Embedded-13 Cursor Declare curs for Select c#, lname, m.title from C, R, M where .... Active set c Single row SELECT does not need a FETCH but result is bound to variables: SELECT a,b FROM... INTO :x,:y WHERE © HS-2009 9-Embedded-15 2 9.4 SQL and Java SQL & Java: JDBC und SQLJ Concepts similar to Embedded SQL, Standardized independent from SQL became popular with Java and the Web protocols Business logic, the application Java-Application Java-Applet or WWW-Browser Http, Corba, RMI Application server JDBC JDBC Proprietary access Protocol over a TCP/IP (?) connection DBMS © HS-2009 DBMS 9-Embedded-24 9.4.1 JDBC 9-Embedded-25 Send SQL strings to be processed, results in a ResultSet object, which is processed with a kind of "hidden cursor" Load a driver (included in java.sql.*), many vendor products Class.forName("oracle.jdbc.driver.OracleDriver"); url is a variable holding the JDBC-Driver and host information Set up the connection to one or more database(s) Connection con = DriverManager.getConnection( "jdbc:oracle:thin:@sunset.inf.fuberlin.de:1521:hstarent", username, password); Several connections at a time may be used. Create a statement object Statement stmt = con.createStatement(); something like a channel, through which queries are sent to the DB Note: this is NOT a statement, but a "statement channel". 9-Embedded-26 JDBC: variable binding, binding, result set iterator Variable binding and result set java.sql.Statement stmt = con.createStatement(); ResultSet r = stmt.executeQuery("SELECT a, b, c FROM Table1"); while (r.next()) { // print the values for the current row. Compare int i = r.getInt("a"); variable binding String s = r.getString("b"); by position float f = r.getFloat("c"); System.out.println("ROW = " + i + " " + s + " " + f); ResultSet rs = stmt.executeQuery("SELECT * FROM Movies WHERE pricePD > 2" ); Process the results one after the other while (rs.next()) { // Loop through each column, get the // column data and display for (i = 1; i <= numCols; i++) { Variable binding if (i > 1) System.out.print(","); by position System.out.print(rs.getString(i)); } } © HS-2009 9-Embedded-27 SQL and Java Excecptions try { // Code that could generate an exception goes here. // If an exception is generated, the catch block below // will print out information about it. } catch(SQLException ex) { System.out.println("\n--- SQLException caught ---\n"); while (ex != null) { System.out.println("Message: " + ex.getMessage ()); System.out.println("SQLState: " + ex.getSQLState ()); System.out.println("ErrorCode: " + ex.getErrorCode ()); ex = ex.getNextException(); System.out.println(""); } } process all execptions No explicit cursor, but iterator using methods boolean next() , void close(), <JavaType> get<JavaType> (), boolean wasNull() to iterate through the result. ... and more in JDBC >= 2.0 © HS-2009 Processing import java.sql.*; © HS-2009 JDBC • Java call-level interface (API) for SQL DBS • DB vendor independent • Supports static and dynamic SQL • Implemented by nearly all DB vendors SQL & Java: JDBC Preparation © HS-2009 SQLJ • essentially embedded SQL for Java • Compiles to JDBC method call, more interveaved with programming langugage. • Defined and implemented by major DBS companies (Oracle in particular) 9-Embedded-28 --- SQLException caught --Message: There is already an object named 'Meier' in the database. Severity 16, State 1, Line 1 SQLState: 42501 -- Defined as X/Open standard ErrorCode: 2714 -- Vendor specific © HS-2009 9-Embedded-29 3 JDBC JDBC Accessing columns The class ResultSet has methods for each type to access result data by position or by name By position: String s = rs.getString(2); // the second attribute to be bound. By name: String rs.getString ("b") ; // get the value of the attribute b of the // row under the (implicit) cursor What about ... Input parameters for queries? (... where attr = :val) Positioned updates without an explicit cursor? Exceptions? © HS-2009 transactional facilities ? 9-Embedded-30 Prepared vs nonnon-prepared Prepared statements Subclass of statement will be compiled, ? Indicates still missing value String mTitle; .... try { java.sql.PreparedStatement prepStmt = con.prepareStatement("SELECT count(*) FROM M,T where M.title = ? and T.mId = M.mId); prepStmt.setString(1, mTitle); ResultSet r = prepStmt.executeQuery() ; while (r.next()) Bind value to position (!) { in statement and excecute int i = r.getInt(1); // must get by position, no name available System.out.println("Number of tapes for " + movieTitle + " is: " +i) } } catch(SQLEception {…} © HS-2009 9-Embedded-31 Very simple performance comparison • Overhead for compiling SQL-statement basically constant • Ratio of compile Time / processing time for queries important • Simple queries: prepare when executed frequently – e.g. in a loop • Complex queries in a loop: no performance gain • Many more factors to be analyzed all queries executed 100 x time: msec System: Postgres local Query: SELECT * FROM city WHERE name ='X' Time: 150 --------------------------System: Postgres local Prepared Query: SELECT * FROM city WHERE name ='X' Time: 60 --------------------------System: Postgres local Query: SELECT DISTINCT A.Name, B.Name FROM country A, country B, encompasses uA, encompasses uB, geo_sea gMA, geo_sea gMB WHERE A.Name < B.Name AND uA.continent= 'Europa' AND uB.continent = 'Europa' ANDuA.country = A.c_ID AND uB.country = B.c_ID AND A.c_ID = gMA.c_ID AND B.c_ID = gMB.c_ID AND gMA.sea = gMB.sea Time: 44802 --------------------------System: Postgres local Prepared Query: SELECT DISTINCT A.Name, B.Name FROM country A, .. Time: 42549 --------------------------- System: Ora9i local Query: SELECT * FROM city WHERE .. Time: 301 --------------------------System: Ora9i local Prepared Query: SELECT * FROM city . Time: 120 --------------------------System: Ora9i local Query: SELECT DISTINCT A.Name, B.Name Time: 1692 --------------------------System: Ora9i local Prepared Query: SELECT DISTINCT A.Name, B.Name FROM Time: 1583 --------------------------- Ora: more effort into optimizing query? © HS-2009 9-Embedded-32 © HS-2009 JDBC: Updates Transactional processing... Updating result sets (JDBC 2.0) ... a short introduction Statement stmt = con.createStatement( ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_UPDATABLE); stmt.setFetchSize(25); ResultSet rs = stmt.executeQuery( "SELECT emp_no, salary FROMemployees"); rs.first(); rs.updateFloat(“salary“, 10000.0); rs.updateRow(); 9-Embedded-33 Transaction: a unit of work which consists of a sequence of steps (operations on the Database) Isolation of independent application processes T1, T2 : T1: read x =3 T2 read x=3 T1: write (x+1) T2 write (x+1) Isolation compromised Lost update! Database corrupt. Why is TA isolation more than well-known concurrency issues in ProgLang? © HS-2009 9-Embedded-35 © HS-2009 9-Embedded-36 4 Transaction semantics Transactions DBS has to guarantees certain executional properties "All or nothing" semantics All effects are made permanent at COMMIT, not before . TA has no effect after ROLLBACK ATOMICITY "Now and forever" DBS guarantees the effects after COMMIT has been processed successfully DURABILITY "Solve concurrency conflicts" Conflict resolution of concurrent operations on DB "Keep consistent DB consistent" ISOLATION Preservation of integrity How does DB System guarantee the properties? Ö Implementation of DBS Application programming: DB transactions Syntactically mark unit of work: START TRANSACTION ……. COMMIT; or: START TRANSACTION ……. IF (everything_OK) COMMIT ELSE ROLLBACK; ENDIF – no effect on DB CONSISTENCY © HS-2009 9-Embedded-37 SQL & Java: JDBC © HS-2009 9-Embedded-38 JDBC: calling stored procedures Transactional properties of connections JDBC constants: TRANSACTION_NONE Different TRANSACTION_READ_UNCOMMITTED isolation levels TRANSACTION_READ_COMMITTED TRANSACTION_REPEATABLE_READ TRANSACTION_SERIALIZABLE JDBC API: • public void setTransactionIsolation(int level) throws SQLExceptionpublic • public void setAutoCommit(boolean autoCommit) -- no autocommit when updating result sets! • public void commit() throws SQLException • public void rollback() throws SQLException © HS-2009 Exception handling if application wants to commit, but DBS cannot guarantee proper execution. 9-Embedded-39 JDBC and persistent objects Stored procedures may be called directly simple CallableStatement cs function expl. = con.prepareCall("{?=call rand(?, 100 ) }"); cs.setInt(2,1); // set arguments cs.registerOutParameter(1, Types.INTEGER); // register type of output cs.execute(); System.out.println( cs.getInt(1) + ", "); // execute and get result value, position 1 If stored procedure causes updates: int i = cs.executeUpdate(); © HS-2009 9-Embedded-42 JDBC connection pooling Persistence of Java Objects ResultSet rs = stmt.executeQuery( "SELECT Employee FROM PERSONNEL"); rs.next(); Employee emp = (Employee)rs.getObject(1); Opening a connection is expensive DB login, set up context, check rights, ... up to 1 sec in some systems Updating objects in the DB emp.setSalary(emp.getSalary() * 1.5); PreparedStatement pstmt = con.preparedStatement( "UPDATE PERSONNEL SET Employee = ? WHERE Employee.no = 1001")); pstmt.setObject(1, emp); pstmt.executeUpdate(); Persistence frameworks: no SQL statements visible for application programmer. Persistent object collections may be updated in java terms emp.setSalary (emp.getSalary*1,5) and updated in the DB "automatically". © HS-2009 9-Embedded-43 Typical interactive applications (e.g. web shop): many customers, few DB interactions ⇒ heavy overhead caused by setup of DB connection Solution use a set of pre-initialized connections ("connection pool") © HS-2009 9-Embedded-44 5 JDBC connection manager JDBC: programming style Do not hard-code connection data but use property file application drivers=jdbc:postgresql jdbc:oracle:thin logfile=D:\\user\\src\\java\\DBConnectionManager\\log.txt DBConnectionManager Connection getConnection(), .... create one or more pools DBConnectionPool //uses vector to store ----------------------------Connection getConnection(), void release(); void init(); // n connections... ora.url=jdbc:oracle:thin:@localhost:1521:tarenths ora.maxconn=2 pg.url=jdbc:postgresql://localhost:5432/geo pg.user=demo different pools pg.password=demopw init() method: Create property object Properties dbProp = new Properties(); access db © HS-2009 e.g. see: http://www.webdevelopersjournal.com/columns/connection_pool.html 9-Embedded-45 JDBC: programming style and Read property file © HS-2009 SQLJ and JDBC // • Separate DB interaction and processing • SQL code goes into a separate object • Encapsulate DB execution and error handling from application processing SQLJ int n; #sql { INSERT INTO emp VALUES (:n)}; // • Many advantages, but not always a trivial task. © HS-2009 9-Embedded-46 9-Embedded-47 9.5 OROR-Mapping JDBC int n; Statement stmt = conn.prepareStatement (“INSERT INTO emp VALUES (?)”); stmt.setInt(1,n); stmt.execute (); stmt.close(); © HS-2009 9-Embedded-49 Goal of OROR-Mapping Motivation Hide details of DB schema from application program typical usage: • application started from scratch • application is rather simple: few persistent classes • Relational schema is generated from persistent classes • Mapping between tables and persistent classes is generated • Most simple approach: – "row type" (attributes) = class – "row" = "object" Systems: Hibernate, Object Relational Bridge (OJB) and many more http://www.systemmobile.com/articles/IntroductionToHibernate.html © HS-2009 9-Embedded-52 Take advantage of the things SQL databases do well, without leaving the Java (?) language of objects and classes. © HS-2009 9-Embedded-53 6 OROR-mapping with Hibernate example OR Mapping Definition From Classes to tables, from rows to objects <class name=“AuctionItem” table=“AUCTION_ITEM”> <id name=“id” column=“ITEM_ID”> <generator class=“native”/> </id> <property name=“description” column=“DESCR”/> <one-to-one name=“successfulBid” column=“SUCCESSFUL_BID_ID”/> <set name=“bids” cascade=“all” lazy=“true”> <key column=“ITEM_ID”/> <one-to-many class=“Bid”/> </set> </class> Mapping has to be defined by developer or generated. © HS-2009 9-Embedded-54 OROR-Mapping Hibernate example © HS-2009 9-Embedded-55 OROR-Mapping Hibernate example Retrieve an AuctionItem and change the description, as detached object: Retrieve an AuctionItem and change the description: Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); AuctionItem item = (AuctionItem) session.get(ActionItem.class, itemId); Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); AuctionItem item = (AuctionItem) session.get(ActionItem.class, itemId); tx.commit(); session.close(); item.setDescription(newDescription); item.setDescription(newDescription); tx.commit(); session.close(); Session session2 = sessionFactory.openSession(); Transaction tx = session2.beginTransaction(); session2.update(item); tx.commit(); reattach! session2.close(); © HS-2009 9-Embedded-56 OROR-Mapping Hibernate Example © HS-2009 9-Embedded-57 Hibernate Query Options Retrieve an AuctionItem and create a new persistent Bid: Bid bid = new Bid() bid.setAmount(bidAmount); Session session=sessionFactory.openSession(); Transaction tx=session.beginTransaction(); AuctionItem item = (AuctionItem)session.get(ActionItem.class, itemId); bid.setItem(item); item.getBids().add(bid); No managed associations! • Hibernate Query Language (HQL) object-oriented dialect of ANSI SQL • Criteria queries (QBC) extensible framework for query objects includes Query By Example (QBC) • Native SQL queries direct passthrough with automatic mapping named SQL queries in metadata tx.commit(); session.close(); © HS-2009 9-Embedded-58 © HS-2009 9-Embedded-59 7 Hibernate Query Language (HQL) OROR-Mapping: Mapping: ● Make SQL “object-oriented” • classes and properties instead of tables and columns • supports polymorphism • automatic association joining • much less verbose than SQL ● Full support for relational operations • inner/outer/full joins, cartesian product • projection, ordering, aggregation and grouping • subqueries and SQL functions Most simple HQL query! © HS-2009 9-Embedded-60 OROR-Mapping Hibernate example Hibernate HQL from AuctionItem; i.e. get all the AuctionItems: List allAuctions = session.createQuery(“from AuctionItem”).list(); © HS-2009 9-Embedded-61 Hibernate Query: select item from AuctionItem item join item.bids as bid where item.description like “Hibernate%” and bid.amount > 100 Criteria / HQL List auctionItems = session.createCriteria(AuctionItem.class).setFetchMode(“bids ”, FetchMode.EAGER) .add( Expression.like(“description”, desc) ) .createCriteria(“successfulBid”) .add( Expression.gt(“amount”, minAmount) ).list(); Equivalent HQL: i.e. get all the AuctionItems with a Bid worth more than 100 and an item description that starts with “Hibernate”. © HS-2009 9-Embedded-62 HQL: Passing Parameters from AuctionItem item left join fetch item.bids where item.description like :description and item.successfulbid.amount > :minAmount © HS-2009 9-Embedded-63 9.5 Components and OROR-Mapping Persistent Architecture : Object-Relational mapping Query q = s.createQuery( "from foo in class Foo where foo.name=:name and foo.size=:size"); user interaction Generated by ORmapping tool q.setProperties(fooBean); // fooBean has getName() and getSize() List foos = q.list(); © HS-2009 9-Embedded-64 © HS-2009 "Business Logic" DB access Object interface Table interface (JDBC) 9-Embedded-65 8 Components and OROR-Mapping Components: EJB Component persistence Architecture Motivation EJB should relieve application from programmer technical issues – multi-user (concurrency management) and distributed user interaction EJB container "Business Logic" DB access transaction management – security – resource management and component lifecycle (threads, database connections, etc.) – remote accessibility and component location transparency – persistence, chaching – tools for bean installation and deploying Object interface Table interface EJB: Enterprise Java Beans, component model for Business related software 9-Embedded-66 © HS-2009 portability: developing business beans without knowing the particular DB infrastructure ⇒ Economic impact! © HS-2009 9-Embedded-68 EJB EJB performance Disadvantage not easy to comprehend not easy to understand side effects e.g. concurrency control in DB or in EJB container • Container: compromises durability ("changed data on disk?") • DB: heavy traffic between EJB cache and DB cache Depend on vendor implementations (EJB used to be underspecified) Performance: heavy penalty depending on application Comparison of JDBC and EJB architectures using TPC-W* How portability is achieved will not be discussed here -> course on J2EE development © HS-2009 9-Embedded-69 TPC-W * Study by Zink / Hettel, Hamburg, AK Frameworks2000 represents Web application (Business model: online book shop) – Browser (Client) – Web-Server EJB using "container – Application-Server managed persistence", i.e. – database automatic mapping between Measurement on DB size: DB objects and EJBs – books: 9.000, Authors: 2.500 – Customers: 5.000, Addresses 9.000, Transactions defined by TPC-W (search, buy, ...) © HS-2009 9-Embedded-70 Summary • Access of Database from application program more important than interactive SQL • cursor : iterator through result sets explicit or implicit • Dynamic versus static (compiled) SQL • Stored procedures: important tool • JDBC : similar to "call level interface (CLI) • SQLJ : "Embedded" version, not as important as JDBC (?) • Persistence frameworks useful in simple applications, questionable in large high performance systems. • Transaction = unit of work: very important concept in multi user environments (remember: acid) • Isolation level: lower levels acceptable in many cases, "seriazalibility" prevents any harm due to read / write conflicts © HS-2009 9-Embedded-76 9