* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download 08_DBApplicationDeve.. - School of Information Technologies
Survey
Document related concepts
Serializability wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Tandem Computers wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Microsoft Access wikipedia , lookup
Ingres (database) wikipedia , lookup
Functional Database Model wikipedia , lookup
Oracle Database wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Concurrency control wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
ContactPoint wikipedia , lookup
Database model wikipedia , lookup
Clusterpoint wikipedia , lookup
Relational model wikipedia , lookup
Transcript
INFO2120 – INFO2820 – COMP5138 Database Systems Week 8: Database Application Development (Kifer/Bernstein/Lewis – Chapter 8; Ramakrishnan/Gehrke – Chapter 6; Ullman/Widom – Chapter 9) Dr. Uwe Röhm School of Information Technologies Outline Database Application Architectures Client-side DB Application Development Call-level Database APIs: PHP/PDO and JDBC Database Application Design Principles Server-side DB Application Development Stored Procedures Based on slides from Kifer/Bernstein/Lewis (2006) “Database Systems” and from Ramakrishnan/Gehrke (2003) “Database Management Systems”, and also including material from Fekete and Röhm. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-2 Database Applications INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-3 Data-intensive Systems Three types of functionality: Presentation Logic - Input – keyboard/mouse - Output – monitor/printer Processing Logic - Business rules - I/O processing Data Management (Storage Logic) - data storage and retrieval GUI Interface Procedures, functions, programs DBMS activities The system architecture determines whether these three components reside on a single system (1-tier) or whether they are distributed across several tiers INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-4 Possible System Architectures 1-Tier Architectures: Centralised Systems 2-Tier Architectures: Client-Server Systems 3-Tier Architectures Client - Server - Middleware Internet Applications Web Databases INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-5 Centralized System centralized system presentation application services services API DBMS user module Presentation Services - displays forms, handles flow of information to/from screen Application Services - implements user request, interacts with DBMS Transactional properties automatic (isolation is trivial) or not required (this is not really an enterprise) DBMS runs within the user process Examples: Access; any application with an integrated DB (e.g. SQLite) – from smartphones to PCs INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-6 Client/Server Model of TPS client machines database server machine ••• presentation application services services DBMS presentation application services services communication / network INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-7 Three-Tiered Model of TPS client machines application / web server machine database server machine ••• presentation server application server DBMS presentation server communication (IPC or network) Presentation Tier Middle Tier INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) Data Management Tier 08-8 Interactive vs. Non-Interactive SQL Interactive SQL: SQL statements input from terminal; DBMS outputs to screen Inadequate for most uses It may be necessary to process the data before output Amount of data returned not known in advance SQL has very limited expressive power (not Turing-complete) Non-interactive SQL: SQL statements are included in an application program written in a host language, like C, Java, COBOL Nowadays also: as embedded in dynamic webpages Client-side vs. Server-side application development Server-side: Stored Procedures and Triggers INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-9 Outline Database Application Architectures Client-side DB Application Development Call-level Database APIs: PHP/PDO Call-level Database API for Java: JDBC Database Application Design Principles Server-side DB Application Development Stored Procedures Based on slides from Kifer/Bernstein/Lewis (2006) “Database Systems” and from Ramakrishnan/Gehrke (2003) “Database Management Systems”, and also including material from Fekete and Röhm. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-10 SQL in Application Code SQL commands can be called from within a host language (e.g., C++ or Java) program. SQL statements can refer to host variables (including special variables used to return status). Must include a statement to connect to the right database. Two main integration approaches: Statement-level interface (SLI) Embed SQL in the host language (Embedded SQL in C, SQLJ) Application program is a mixture of host language statements and SQL statements and directives Call-level interface (CLI) Create special API to call SQL commands (JDBC, ODBC, PHP, …) SQL statements are passed as arguments to host language (library) procedures / APIs INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-11 Call-level Interfaces and Database APIs Rather than modify compiler, add library with database calls (API) Special standardized interface: procedures/objects Pass SQL strings from language,present result sets in language-friendly way Supposedly DBMS-neutral a “driver” executes the calls and translates them into DBMS-specific code database can be across a network Several Variants SQL/CLI: “SQL Call-Level-Interface” Part of the SQL-92 standard; “The assembler under the APIs” JDBC, ODBC, PDO, … ODBC: “Open DataBase Connectivity” Side-branch of early version of SQL/CLI Enhanced to: OLE/db, and further ADO.NET JDBC: “Java DataBase Connectivity” Native Interface CLI Java standard PDO DBMS Persistency standard for PHP Data Objects INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-12 PDO – PHP Data Objects INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-13 PHP PHP is a scripting language for dynamic websites PHP – original recursive acronym for "PHP: Hypertext Preprocessor” embedded into HTML Indicated by <?php PHP-code ?> There are several different approacheson how to connect in PHP scripts to databases Vendor-specific database extensions e.g. pgsql (PostgreSQL) or pci8 (Oracle) => Outdated! Some abstraction layers on top (typically for PHP 5.1 onwards) e.g. PDO (“PHP Data Objects”) Generic DB library also via PEAR (PHP Extension&Application Repository) INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-14 PHP 101 A dynamically-typed scripting language Embedded in normal HTML page Offers the usual programming constructs: Variable Condition statements Loops Input/output Example (example.php): <html> <head><title>PHP Test</title></head> <body> <h1>This is a PHP test</h1> Today is <?php echo "a just normal day" ?>, the <?php echo date("F j, Y") ?>. </body> </html> INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-15 PHP 101: Variables in PHP Must begin with $ Dynamically typed – it is OK to not declare a type for a variable. But you give a variable a value that belongs to a “class,” in which case, methods of that class are available to it. String Variables: PHP solves a very important problem for languages that commonly construct strings as values: How do I tell whether a substring needs to be interpreted as a variable and replaced by its value? PHP solution: Double quotes means replace; single quotes means don’t. $100 = ”one hundred dollars”; $sue = ’You owe me $100.’; $joe = ”You owe me $100.”; Value of $sue is ’You owe me $100’, while the value of $joe is ’You owe me one hundred dollars’. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-16 PHP 101: Array Variables in PHP Two kinds: numeric and associative. Numeric arrays are ordinary indexed 0,1,… Example: $a = array("Paul", "George", "John", "Ringo"); Then $a[0] is "Paul", $a[1] is "George", and so on. Elements of an associative array $a are pairs x => y, where x is a key string and y is any value. If x => y is an element of $a, then $a[x] is y. Example: $a = array("bass" => "Paul", "guitar" => "George", "guitar2"=>"John", "drums" => "Ringo"); Then $a[‘bass’] is "Paul", $a[‘drums’] is "Ringo", and so on. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-17 PDO – PHP Data Objects Introduced since PHP 5.1 (in 2005) Object-oriented extension to PHP for database programming that provides a database abstraction layer Generic driver model to connect to different database engines via the same API Significant improvement over the previous proprietary APIs URL: http://www.php.net/manual/en/intro.pdo.php INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-18 PDO Example <?php function printClassList ($unit_of_study, $user, $pwd) { try { /* connect to the database */ $conn=new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pwd); /* prepare a dynamic query */ $stmt = $conn->prepare('SELECT name FROM Student NATURAL JOIN Enrolled WHERE uosCode = :uos'); $stmt->bindValue( ':uos', $unit_of_study, PDO::PARAM_STR, 8 ); /* execute the query and loop through the resultset */ $results = $stmt->execute(); while ( $row = $results->fetch() ) { print " student: ", $row['name']; } /* clean up */ $stmt->closeCursor(); $conn = null; } ?> } catch (PDOException $sqle) { /* error handling */ print "SQL exception : ", $sqle->getMessage(); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-19 Core Problems with SQL Interfaces (1) Establishing a database connection (2) Static vs. Dynamic SQL (3) Mapping of domain types to data types of host Concept of host variable How to treat NULL values? (4) Impedance Mismatch: SQL operates on sets of tuples Host languages like C do not support a set-of-records abstraction, but only a one-value-at-a-time semantic Solution: Cursor Concept Iteration mechanism (loop) for processing a set of tuples (5) Error handling INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-20 (1): PDO Run-Time Architecture Oracle Oracle database driver PHP code PDO PostgreSQL PostgreSQL driver database MySQL driver ... MySQL DBMS database PDO is DBMS independent PDO functions are generic PDO allows to connect to specific driver Using parameters of PDO constructor Even to different databases from the same program Database drivers are loaded and used at run-time INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-21 PDO Connections Session with a data source started by creating a PDO object: $conn = new PDO( DSN, $userid, $passwd [,$params] ); Data Source Name (DSN) of the form <driver>:<connectionParameter1>;<connectionParameter2>;… For example with PostgreSQL: $conn = new PDO( "pgsql:host=postgres.it.usyd.edu.au;dbname=unidb",$user,$pw); driver connectionParameters db login Details: http://www.php.net/manual/en/pdo.construct.php INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-22 PDO Connection Drivers Driver support for variety of DBMSs MySQL (prefix: mysql) Note: PostgreSQL (prefix: pgsql) drivers need to be Oracle (prefix: oci) installed first as part of the PHP server's IBM DB2 (prefix: ibm) configuration… SQL Server (prefix: sqlsrv) sqlite (prefix: sqlite) … DSN syntax and additional DB parameters vary for each driver Check manuals: http://www.php.net/manual/en/pdo.drivers.php Example for Oracle: $conn = new PDO( "oci:dbname=oracle10g.it.usyd.edu.au:1521/ORCL", $user,"Database $pwdSystems" ); - 2013 (U. Röhm) INFO2120/INFO2820/COMP5138 08-23 PDO Connection Example <?php try { /* connect to the database */ $conn = new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pw); /* query database */ $stmt = $conn->query('SELECT name FROM Student WHERE studID=4711'); … Do Actual Work …. /* clean up */ $stmt->closeCursor(); $conn = null; ?> } /* error handling */ catch (PDOException $sqle) { print "SQL exception : ", $sqle->getMessage(); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-24 PDO Objects PDO __construct(…) query() prepare() beginTransaction() commit() rollBack() … PDOStatement query(stmt) prepare(stmt) PDOException array $errorInfo getMessage() getPrevious() getCode() getFile() getLine() … INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) string $queryString bindValue() bindParam() bindColumn() execute() fetch() fetchColumn() fetchAll() nextRowset() closeCursor() errorCode() … 08-25 PDO Class Interface Start SQL statements query() for static SQL, or prepare() for parameterized SQL queries exec() for immediately executing some SQL; returns num rows Transaction control beginTransaction() commit() rollBack() inTransaction() starts a database transaction (otherwise: autocommit) successfully finishes current transaction aborts current transaction checks whether there's an active transaction Sets/gets connection parameters (often driver specific) getAttribute(…) setAttribute(…) Error Handling errorCode() errorInfo() [cf. http://www.php.net/manual/en/class.pdo.php] INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-26 Side Note on DB Connections Establishing a database connection takes some time… Network communication, memory allocation, dbs authorization So do this only once in your program … but not for individual SQL queries Modern, multi-threaded applications will typically want to have a pool of connections that are re-used Might be handled by your runtime library (that's what happens in PHP) But for, e.g., Java programs better be mindful of connection costs! INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-27 (2) Executing SQL Statements Three different ways of executing SQL statements: PDOStatement PDO::query(sql) semi-static SQL statements PDOStatement PDO::prepare(sql) parameterized SQL statements num_rows PDO::exec(sql) immediately run SQL command PDOStatement class: Precompiled, parameterized SQL statements: Structure is fixed after call to PDO::prepare() Values of parameters are determined at run-time Fetch and store routines are executed when PDOStatement::execute() is executed to communicate argument values with DBMS PDOStatement::execute() can be invoked multiple times with different values of in parameters Each invocation uses same query execution plan INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-28 PDOStatement with Semi-static SQL Simplest way to execute some static SQL query: <?php try { /* connect to the database */ … This is 'semi-static' because one could construct the SQL string during runtime. Warning: DON'T DO THIS! Use parameterized queries instead! (cf. SQL Injection problem later) /* query database */ $stmt = $conn->query('SELECT name FROM Student WHERE studID=4711'); $name = $stmt->fetchColumn(); /* just fetch the single return value */ print $name; /* clean up */ $stmt->closeCursor(); ?> } /* error handling */ catch (PDOException $sqle) { print "SQL exception : ", $sqle->getMessage(); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-29 Static vs. Dynamic SQL SQL constructs in an application can take two forms: Standard SQL statements (static embedded SQL): Useful when SQL portion of program is known at compile time Only available with Embedded SQL in compiled language… Directives (dynamic SQL): Useful when SQL portion of program not known at compile time. Application constructs SQL statements at run time as values of host language variables that are manipulated by directives. Problem is: PHP is not a compiled language; So everything in PHP/PDO is by definition dynamic SQL… Still: Try to avoid constructing SQL strings in the program from user input, rather use fixed query structures with parameters (parameterized queries) INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-30 Approach 2: Preparing and Executing a parameterized Query $query = "SELECT E.studId FROM Enrolled E WHERE E.uosCode = ? AND E.semester = ?"; placeholders $stmt = $conn->prepare ( $query ); • Prepares the statement • Creates a prepared statement object, $stmt, containing the prepared statement • Placeholders (?) mark positions of in parameters; special API is provided to plug the actual values in positions indicated by the ?’s INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-31 Preparing & Executing a Query (cont’d) var $uos_code, $semester; ……… $stmt->bindValue(1, $uos_code); // set value of first in parameter $stmt->bindValue(2, $semester); // set value of second in parameter $stmt->execute (); • Evaluates parameters bound with setParameter() only now • Executes the query • Associates a result set with the same PDOStatement while ( $row = $stmt->fetch ( ) ) { $j = $row['studId']; …process output value… } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) // advance the cursor // fetch output int-value 08-32 (3) Host Variables Data transfer between DBMS and application Mapping of SQL domain types to data types of host language PHP PDO: Host variables are normal mixed PHP variables that are dynamically typed and accessed during runtime: $studid = 12345; $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=?"); $stmt->bindValue(1, $studid); Note: in statement-level APIs such as ESQL/C: Host variables must be declared before usage EXEC SQL int char EXEC SQL BEGIN DECLARE SECTION; studid = 12345; sname[21]; END DECLARE SECTION; INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) Variables shared by host and SQL 08-33 PDO: Parameterized Queries Two Approaches for specifying query parameters: 1. Anonymous Placeholders $studid = 12345; $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=?"); $stmt->bindValue(1, $studid); 2. Named Placeholders $studid = 12345; $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=:s"); $stmt->bindValue(':s', $studid); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-34 PDO: Binding Host Variables Two Approaches for binding host variables as input params: PDOStatement::bindValue() binds value of host variable at call PDOStatement::bindParam() binds host variable by reference Example $studid = 12345; $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=:s"); $stmt->bindParam(':s', $studid); $studid = 56789; $stmt->execute(); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-35 PDO: Typing Host Variables Host variables can be dynamically typed $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=:s"); $stmt->bindValue(':s', 12345); or type-safe with (optional) third type parameter PDO::PARAM_INT PDO::PARAM_STR PDO::PARAM_BOOL PDO::PARAM_LOB PDO::PARAM_NULL represents an SQL INTEGER represents a SQL CHAR or VARCHAR represents a boolean represents a SQL large object data type represents SQL NULL Example: $studid = 12345; $stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=:s"); $stmt->bindValue(':s', $studid, PDO::PARAM_INT); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-36 PDO: Binding Output Variables For binding output parameters: PDOStatement::bindColumn() binds a output column to a PHP var PDOStatement::fetch(PDO::FETCH_BOUND) fetches values into vars Can also be strongly typed during bindColumn() call Example: $sql = "SELECT name,gender,address FROM Student WHERE sid=4711"; $stmt= $conn->prepare($sql); $stmt->execute(); /* option 1: bind by column number */ $stmt->bindColumn(1, $name, PDO::PARAM_STR); $stmt->bindColumn(2, $gender, PDO::PARAM_STR ); /* option 2: bind by column name */ $stmt->bindColumn('address', $addr); $row = $stmt->fetch(PDO::FETCH_BOUND); print $name, '\t',$gender, '\t',$addr, '\n'; INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-37 Preparing & Executing Dynamic Updates $sql="INSERT INTO Student VALUES(?,?,?,?)"; $pstmt = $conn->prepare($sql); $pstmt.bindValue(1, $sid, PDO::PARAM_INT); $pstmt.bindValue(2, $sname, PDO::PARAM_STR); $pstmt.bindValue(3, $birthdate, PDO::PARAM_STR); $pstmt.bindValue(4, $country, PDO::PARAM_STR); /* execute with latest values from host variables */ $pstmt.execute(); $numRows1 = $pstmt.rowCount(); /* execute again with dynamically bound values */ $pstmt.execute( array(1234,'Obama',NULL,'USA') ); $numRows2 = $pstmt.rowCount(); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-38 (4) Buffer Mismatch Problem (also: Impedance Mismatch) Problem: SQL deals with tables (of arbitrary size); host language program deals with fixed size buffers How is the application to allocate storage for the result of a SELECT statement? Solution: Cursor concept Fetch a single row at a time cursor application SELECT Result set (or pointers to it) INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) Base table 08-39 Mapping of Sets: Cursor Concept Result set – set of rows produced by a SELECT statement Cursor – pointer to a row in the result set. Cursor operations: Declaration Open – execute SELECT to determine result set and initialize pointer Fetch – advance pointer and retrieve next row (JDBC: next() call) Close – deallocate cursor INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-40 Cursor in PDO – via PDOStatement Cursor concept with PHP/PDO: $stmt = $conn->prepare("SELECT title,name,address FROM Emp"); $stmt->execute(); while ( $row = $stmt->fetch() ) { $data = $row[0] . "\t" . $row[1] . "\t" . $row[2] . "\n"; print $data; You can address } result columns either $stmt->closeCursor(); by name or position PHP language natively supports arrays; good for small results $stmt->execute(); $resultset = $stmt->fetchAll(); foreach ( $resultset as $row ) { print_r($row); } just be mindful that this can be VERY memory hungry for large results INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-41 PDOStatement::fetch() mixed PDOStatement::fetch ( [ int $fetch_style [, int $cursor_orientation = PDO::FETCH_ORI_NEXT [, int $cursor_offset = 0 ]]] ) where $fetch_style Controls how new result row will be returned to caller PDO::FETCH_ASSOC as an associative array PDO::FETCH_NUM as numerically-index array, starting at 0 PDO::FETCH_BOTH both of above (DEFAULT) PDO::FETCH_BOUND fetch in bound output column variables … $cursor_orientation Whether it is a scrollable cursor, or not (DEFAULT) $cursor_offset for a scrollable cursor, the absolute row number to fetch first INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-42 NULL Handling in PDO Remember: Null values mean neither 0 nor empty string Hence special indication of unknown values needed. In PHP this is quite natural, as PHP supports NULL: $stmt = $conn->query("SELECT gender FROM Student …"); $row = $stmt->fetch(); if ( is_null($row['gender']) ) { /* null value */ } else { /* no null value */} Other languages require a special indicator variable. Eg. C: EXEC SQL select gender into :gender:indicator from Student where sid=4711; if ( indicator == -1 ) { /* null value */ } else { /* no null value */} INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-43 PHP: isset() vs. empty() vs. is_null() isset(var) empty(var) Returns TRUE if var exists and is not NULL, otherwise returns FALSE. Returns FALSE if var exists and has a nonempty, non-zero value, otherwise TRUE. [http://php.net/manual/en/function.isset. php] [http://php.net/manual/en/function.emp ty.php] is_null(var) Returns TRUE if var === NULL, otherwise FALSE [http://php.net/manual/en/f unction.is-null.php] INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) http://techtalk.virendrachandak.com/php-isset-vs-empty-vs-is_null/ 08-44 NULL Handling in PDO (cont'd) In PDO, the NULL behaviour can be further configured PDO connection attribute PDO::ATTR_ORACLE_NULLS (available with all drivers, not just Oracle): PDO::NULL_NATURAL no conversion. PDO::NULL_EMPTY_STRING empty string is converted to NULL. PDO::NULL_TO_STRING NULL is converted to an empty string. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-45 (5) Error Handling Multitude of potential problems No database connection or connection timeout Wrong login or missing privileges SQL syntax errors Empty results NULL values … Hence always check database return values, Provide error handling code, resp. exception handlers Gracefully react to errors or empty results or NULL values NEVER show database errors to end users Not only bad user experience, but huge security risk… INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-46 You should avoid this! ACM Order Rectification The web site you are accessing has experienced an unexpected error. Please contact the website administrator. The following information is meant for the website developer for debugging purposes. Error Occurred While Processing Request Element ORDERID is undefined in URL. The error occurred in D:\wwwroot\Public\rectifyCC\rectifyCC.cfm: line 463 461 462 463 464 465 : : : : : WHERE a.order_id = b.order_id AND a.order_id = c.order_id AND a.order_id = '#URL.orderID#' </CFQUERY> Resources: Check the ColdFusion documentation to verify that you are using the correct syntax. Search the Knowledge Base to find a solution to your problem. Browser Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en - us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 Remote 129.78.220.7 Address Referrer Date/Time 26- Aug- 09 10:24 PM Stack Trace at cfrectifyCC2ecfm1287160776.runPage(D:\wwwroot\Public\rectifyCC\rectifyCC.cfm:463) Also cf. error #... Of http://www.sans.org/top25-software-errors/ INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-47 Error Handling with PDO Two mechanism: 1. Explicitly testing for error codes after each statement Both PDO and PDOStatement objects provide error status functions: errorCode() fetches the SQLSTATE of last statement errorInfo() fetches extended error information of last stmt. 2. Error handling via normal exception mechanism of PHP This has to be configured on a connection (PDO) object via PDO::setAttribute() PDO::ATTR_ERRMODE: Error reporting. PDO::ERRMODE_SILENT: Just set error codes. PDO::ERRMODE_WARNING: Raise E_WARNING. PDO::ERRMODE_EXCEPTION: Throw exceptions. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-48 SQLSTATE a five characters alphanumeric identifier defined in SQL-92 Two characters error class value Followed by a three characters sub-class value Examples: 00000 successful completion Class 01 indicates a warning eg. 01004 Warning: string data, right truncation or 01007 Warning: privilege not granted Class 02: no data error (SQLSTATE: 02000) Class 08: connection error eg. 08001 Error: unable to establish SQL connection … List of available SQLSTATEs: http://docstore.mik.ua/orelly/java-ent/jenut/ch08_06.htm INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-49 Exception Handling with PDO Class PDOException PDOException::getMessage() returns exception message PDOException::getCode() returns the exception code … Example: 1. Configure to have thrown exceptions on SQL errrors $dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); 2. Catch-Try block around PDO statements: try { … } catch ( PDOException $ex ) { print ex.getMessage(); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-50 Cf. Example in PDO <?php function printClassList ($unit_of_study, $user, $pwd) { try { /* connect to the database */ $conn=new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pwd); /* prepare a dynamic query */ $stmt = $conn->prepare('SELECT name FROM Student NATURAL JOIN Enrolled WHERE uosCode = :uos'); $stmt->bindParam( ':uos', $unit_of_study, PDO::PARAM_STR, 8 ); /* execute the query and loop through the resultset */ $results = $stmt->execute(); while ( $row = $results->fetch() ) { print " student: ", $row['name']; } Host variable concept cursor concept /* clean up */ $stmt->closeCursor(); $conn = null; } ?> } catch (PDOException $sqle) { /* error handling */ print "SQL exception : ", $sqle->getMessage(); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) error handling 08-51 Time for a Break… INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) The following part is meant as background reading for students doing the assignment in Java/JDBC – such as Postgraduate students from COMP5138… JDBC Java Database Connectivity INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-53 JDBC - “Java Database Connectivity” JDBC is a Java API for communicating with database systems supporting SQL JDBC supports a variety of features for querying and updating data, and for retrieving query results JDBC also supports metadata retrieval, such as querying about relations present in the database and the names and types of relation attributes Model for communicating with the database: Open a connection Create a “statement” object Execute queries using the Statement object to send queries and fetch results Exception mechanism to handle errors INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-54 JDBC Example import java.sql.*; public void printLecturerName ( String unit_of_study, String user, String pwd) { try {/* connect to the database */ Class.forName ("org.postgresql.Driver"); Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd); /* prepare the dynamic query */ PreparedStatement stmt = conn.prepareStatement( “select name from Student natural join Enrolled where uosCode=?”); stmt.setString(1, unit_of_study); /* execute the query and loop through the resultset */ ResultSet rset = stmt.executeQuery(); while ( rset.next() ) { System.out.println(“ student: “ + rset.getString(1)); } /* clean up */ stmt.close(); conn.close(); } } catch (SQLException sqle) { /* error handling */ System.out.println("SQLException : " + sqle); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-55 (1): JDBC Run-Time Architecture Oracle Oracle database driver application driver manager SQLServer SQLServer driver database PostgreSQL driver JDBC is DBMS independent ... PostgreSQL DBMS database JDBC functions are generic DriverManager allows to connect to specific driver Even to different databases from the same program Database drivers are loaded and used at run-time JDBC was one of the first APIs giving this flexibility and a lot of effort was put into making this as flexible as possible also during runtime. Hence one indirection more than with PHP/PDO and also more effort to include legacy (non-Java) drivers. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-56 JDBC Architecture Four architectural components: Application (initiates and terminates connections, submits SQL statements) Driver manager (loads JDBC driver during runtime) Note: This part is not explicitly present with PHP/PDO as with PHP, the drivers have to be pre-configured as part of the PHP configuration Driver (connects to data source, transmits requests and returns/translates results and error codes) Data source (processes SQL statements) INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-57 JDBC Driver Management Steps to submit a database query: Load the JDBC driver (during runtime as part of the program) Connect to the data source Execute SQL statements All drivers are managed by the DriverManager class Loading a JDBC driver (variants): Class.forName(driver_class_name) For example for PostgreSQL: Class.forName(“org.postgresql.Driver”); or example for Oracle: Class.forName(“oracle.jdbc.driver.OracleDriver”); When starting the Java application: -Djdbc.drivers=org.posgresql INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-58 JDBC Connections A session with a data source is started through the creation of a Connection object Via the DriverManager: DriverManager.getConnection(DB_URL,userid,passwd); Database URL of the form jdbc:<subprotocol>:<connectionParameters> For example with PostgreSQL: Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd); subprotocol connectionParameters INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-59 Example JDBC Code import java.sql.*; public static void JDBCexample( String user, String pwd ) { try { Class.forName ("org.postgresql.Driver"); Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd); Statement stmt = conn.createStatement(); … Do Actual Work …. stmt.close(); conn.close(); } catch (SQLException sqle) { System.out.println("SQLException : " + sqle); } } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-60 Connection Class Interface Sets isolation level for the current connection. public int getTransactionIsolation() and void setTransactionIsolation(int level) Specifies whether transactions in this connection are readonly public boolean getReadOnly() and void setReadOnly(boolean b) If autocommit is set, then each SQL statement is considered its own transaction. Otherwise, a transaction is committed using commit(), or aborted using rollback(). public boolean getAutoCommit() and void setAutoCommit(boolean b) Checks whether connection is still open. public boolean isClosed() INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-61 (2) Executing SQL Statements Three different ways of executing SQL statements: Statement (both static and dynamic SQL statements) PreparedStatement (semi-static SQL statements) CallableStatement (stored procedures) PreparedStatement class: Precompiled, parameterized SQL statements: Structure is fixed Values of parameters are determined at run-time Fetch and store routines are executed at client when EXECUTE is executed to communicate argument values with DBMS EXECUTE can be invoked multiple times with different values of in parameters Each invocation uses same query execution plan INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-62 Preparing and Executing a Query String query = “SELECT E.studId FROM Enrolled E” + “WHERE E.uosCode = ? AND E.semester = ?”; placeholders PreparedStatement ps = con.prepareStatement ( query ); • Prepares the statement • Creates a prepared statement object, ps, containing the prepared statement • Placeholders (?) mark positions of in parameters; special API is provided to plug the actual values in positions indicated by the ?’s INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-63 Preparing & Executing a Query (cont’d) String uos_code, semester; ……… ps.setString(1, uos_code); // set value of first in parameter ps.setString(2, semester); // set value of second in parameter ResultSet res = ps.executeQuery ( ); • Creates a result set object, res • Executes the query • Stores the result set produced by execution in res while ( res.next ( ) ) { j = res.getInt (“studId”); …process output value… } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) // advance the cursor // fetch output int-value 08-64 (3) Host Variables Data transfer between DBMS and application Mapping of SQL domain types to data types of host language JDBC: Host variables are normal Java variables that are accessed using specific, strongly-typed functions. Example: int studid = 12345; Statement stmt = con.Statement( “SELECT name FROM Student WHERE sid=?”); stmt.setInt(1, studid); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-65 Preparing & Executing Dynamic Updates String sql=“INSERT INTO Student VALUES(?,?,?,?)”; PreparedStatment pstmt=con.prepareStatement(sql); pstmt.clearParameters(); pstmt.setInt(1,sid); pstmt.setString(2,sname); pstmt.setDate(3, new java.sql.Date(birthdate)); pstmt.setString(4, country); // we know that no rows are returned, thus we use executeUpdate() int numRows = pstmt.executeUpdate(); Note: PreparedStatement.executeUpdate only returns the number of affected records INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-66 (4) JDBC: ResultSets PreparedStatement.executeQuery returns data, encapsulated in a ResultSet object (a cursor) ResultSet rs=pstmt.executeQuery(sql); // rs is now a cursor while (rs.next()) { // process the data } rs.close() A ResultSet is a very powerful cursor: previous(): moves one row back absolute(int num): moves to the row with the specified number relative (int num): moves forward or backward first() and last() wasNull() dealing with NULL values INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-67 Matching Java and SQL Types SQL Type Java class ResultSet get method BIT Boolean getBoolean() CHAR String getString() VARCHAR String getString() DOUBLE Double getDouble() FLOAT Double getDouble() INTEGER Integer getInt() REAL Double getFloat() DATE java.sql.Date getDate() TIME java.sql.Time getTime() TIMESTAMP java.sql.TimeStamp getTimestamp() INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-68 NULL Handling in JDBC Remember: Null values mean neither 0 nor empty string Hence special indication of unknown values needed JDBC: wasNull() call for individual columns on ResultSet Embedded SQL in C etc.: null-indicator variable Example: EXEC SQL select name into :sname:indicator from Student where sid=:studid; if ( indicator == -1 ) { /* null value */ } else { /* no null value */} INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-69 (5) JDBC Error Handling: Exceptions and Warnings Most of java.sql can throw and SQLException if an error occurs. SQLWarning is a subclass of SQLException; not as severe (they are not thrown and their existence has to be explicitly tested) try { stmt=con.createStatement(); warning=con.getWarnings(); while(warning != null) { // handle SQLWarnings; warning = warning.getNextWarning(); } con.clearWarnings(); stmt.executeUpdate(queryString); warning = con.getWarnings(); … } //end try catch( SQLException SQLe) { // handle the exception } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-70 Cf. Example in JDBC import java.sql.*; public void printLecturerName ( String unit_of_study, String user, String pwd) { try {/* connect to the database */ Class.forName ("org.postgresql.Driver"); Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd); /* prepare the dynamic query */ PreparedStatement stmt = conn.prepareStatement( “select name from Student natural join Enrolled where uosCode=?”); Host variable stmt.setString(1, unit_of_study); /* execute the query and loop through the resultset */ ResultSet rset = stmt.executeQuery(); while ( rset.next() ) { System.out.println(“ student: “ + rset.getString(1)); } concept cursor concept /* clean up */ stmt.close(); conn.close(); } } catch (SQLException sqle) { /* error handling */ System.out.println("SQLException : " + sqle); } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) error handling 08-71 JDBC: Access to Database Metadata The class DatabaseMetaData provides information about database relations Has functions for getting all tables, all columns of the table, primary keys etc. E.g. to print column names and types of a relation DatabaseMetaData dbmd = conn.getMetaData( ); ResultSet rs = dbmd.getColumns( null, “UNI-DB”, “Student”, “%” ); //Arguments: catalog, schema-pattern, table-pattern, column-pattern // Returns: 1 row for each column, with several attributes such as // COLUMN_NAME, TYPE_NAME, etc. while ( rs.next( ) ) { System.out.println( rs.getString(“COLUMN_NAME”) , rs.getString(“TYPE_NAME”); } There are also functions for getting information such as Foreign key references in the schema Database limits like maximum row size, maximum no. of connections, etc INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-72 This Week’s Agenda Database Application Architectures Client-side DB Application Development Call-level Database APIs: PDO and JDBC Database Programming Design Principles Server-side DB Application Development Stored Procedures INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-73 Design Principles for DB Applications For larger project, the correct ‘layering’ of an app is crucial Presentation layer Business logic Data access layer Data management cf. Model-Viewer-Control (MVC) principle General Design Principles: Separate Data Access Layer and the remaining application logic Dynamic web-languages such as PHP are very tempting in this respect, but horrible to maintain, extend or simply keep secure! Rather: all database access logic should be in its own dedicated data access object and data source wrapping module Do proper error handling don’t expose internal database error messages Validate any user input; use dynamic SQL with parameter parsing Secure your code against SQL injection attacks INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-74 FOR IMMEDIATE RELEASE Monday, August 17, 2009 WWW.USDOJ.GOV CRM (202) 514-2007 TDD (202) 514-1888 Alleged International Hacker Indicted for Massive Attack on U.S. Retail and Banking Networks Data Related to More Than 130 Million Credit and Debit Cards Allegedly Stolen WASHINGTON – Albert Gonzalez, 28, of Miami, Fla., was indicted today for conspiring to hack into computer networks supporting major American retail and financial organizations, and stealing data relating to more than 130 million credit and debit cards, announced Assistant Attorney General of the Criminal Division Lanny A. Breuer, Acting U.S. Attorney for the District of New Jersey Ralph J. Marra Jr. and U.S. Secret Service Assistant Director for Investigations Michael Merritt. In a two-count indictment alleging conspiracy and conspiracy to engage in wire fraud, Gonzalez, AKA "segvec," "soupnazi" and "j4guar17," is charged, along with two unnamed co-conspirators, with using a sophisticated hacking technique called an "SQL injection attack," which seeks to exploit computer networks by finding a way around the network’s firewall to steal credit and debit card information. Among the corporate victims named in the indictment are Heartland Payment Systems, a New Jersey-based card payment processor; 7-Eleven Inc., a Texas-based nationwide convenience store chain; and Hannaford Brothers Co. Inc., a Maine-based supermarket chain. The indictment, which details the largest alleged credit and debit card data breach ever charged in the United States, alleges that beginning in October 2006, Gonzalez and his co-conspirators researched the credit and debit card systems used by their victims; devised a sophisticated attack to penetrate their networks and steal credit and debit card data; and then sent that data to computer servers they operated in California, Illinois, Latvia, the Netherlands and Ukraine. The indictment also alleges Gonzalez and his coconspirators also used sophisticated hacker techniques to cover their tracks and to avoid detection by antivirus software used by their victims. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-77 Anonymous speaks: the inside story of the HBGary hack By Peter Bright | Last updated about a month ago It has been an embarrassing week for security firm HBGary and its HBGary Federal offshoot. HBGary Federal CEO Aaron Barr thought he had unmasked the hacker hordes of Anonymous and was preparing to name and shame those responsible for co-ordinating the group's actions, including the denial-of-service attacks that hit MasterCard, Visa, and other perceived enemies of WikiLeaks late last year. Hacker Gains Access To WordPress.com Servers, Site Source Code Exposed Alexia Tsotsis When Barr told one of those he believed to be an Anonymous ringleader about his forthcoming exposé, the Anonymous response was swift and humiliating. HBGary's servers were broken into, its e-mails pillaged and published to the world, its data destroyed, and its website defaced. As an added bonus, a second site owned and operated by Greg Hoglund, owner of HBGary, was taken offline and the user registration database published. Over the last week, I've talked to some of those who participated in the HBGary hack to learn in detail how they penetrated HBGary's defenses and gave the company such a stunning black eye—and what the HBGary example means for the rest of us mere mortals who use the Internet. Anonymous: more than kids WordPress.com has revealed that someone has gained root - access (“low - level,” as in deep) to several of its servers this morning and that VIP customers’ source code was accessible. WordPress.com VIP customers are all on “code red” and in the process of changing all the passwords/ API keys they’ve left in the source code. “Tough note to comm unicate today: Automattic had a low- level (root) break - in to several of our servers, and potentially anything on those servers could have been revealed. We have been diligently reviewing logs and records about the break - in to determine the ex tent of the inform ation ex posed, and re- securing avenues used to gain access. We presume our source code was ex posed and copied. While much of our code is Open Source, there are sensitive bits of our and our partners’ code. Beyond that, however, it appears information disclosed was lim ited.” While Automattic is downplaying the leak, sites’ source code could include API keys and Twitter and Facebook passwords which can let interested parties gain access to sensitive information as well as shut people out of their Twitter and other vulnerable accounts. The HBGary saga: Autom attic says that the investigation “is ongoing.” I’ve contacted founder Matt Mullenweg for more information and will update this post when I hear back. Anonymous to security firm working with FBI: "You've angered the hive" How one security firm tracked down Anonymous— and paid a heavy price (Virtually) face to face: how Aaron Barr revealed himself to Anonymous Spy games: Inside the convoluted plot to bring down WikiLeaks Apr 13, 2011 WordPress.com currently serves 18 million publishers, including VIPs like us, TED, CBS and is responsible for 10% of all websites in the world. WordPress.com itself sees about 300 million unique visits monthly. Global websites Press About us Contact us Advertisem ent Products Solutions Support Security Partners Anonymous speaks: the inside story of the HBGary hack Black ops: How HBGary wrote backdoors for the government Now Com m ented Facebook Peter Thiel: We’re in a Bubble and It’s Not the Internet. It’s Higher Education. HBGary and HBGary Federal position themselves as experts in computer security. The companies offer both software and services to both the public and private sectors. On the software side, HBGary has a range of computer forensics and malware analysis tools to enable the detection, isolation, and analysis of worms, viruses, and trojans. On the services side, it offers expertise in implementing intrusion detection systems and secure networking, and performs vulnerability assessment and penetration testing of systems and software. A variety of three letter agencies, including the NSA, appeared to be in regular contact with the HBGary companies, as did Interpol, and HBGary also worked MySQL.com and with well-known security firm McAfee. At one time, even Apple expressed an interest in the company's products or services. SQL injection Twitter: Consider This Your Intervention. Please God, All I Want Is A Phone. Any Phone. Google On Its “Amazing Blazingly” Mobile Business: “We Tripped Into $ 1 Billion” Google Ditches YouTube, Goes Back To Relying On Nemesis Microsoft For Earnings Calls Topics SunRelated hacked through wordpress.com Greg Hoglund's rootkit.com is a respected resource for discussion and analysis of rootkits (software that tampers with If you're new here, you might want to operating systems at a low level to evade detection) and related technology; over the years,Hi histhere! site has been targeted updates. by disgruntled hackers aggrieved that their wares have been discussed, dissected, and often disparaged as badly written bits of code. Advertisem ent subscribe to the RSS feed for by Chester Wisniewski on March 27, 2011 | Be the first to comment One might think that such an esteemed organization would prove an insurmountable challenge forData a bunch of FILED UNDER: loss, Featured , Vulnerability disaffected kids to hack. World-renowned, government-recognized experts against Anonymous? HBGary should be Proving that no website is ever truly secure, it is being able to take their efforts in stride. reported that MySQL.com has succumbed to a SQL injection attack. It was first disclosed to the Full Advertisem ent Powered by WordPress.com VIP Unfortunately for HBGary, neither the characterization of Anonymous nor the assumption of competence Disclosure mailing on listthe early this morning. Hackers have Advertise Archives Contact Events security company's part are accurate, as the story of how HBGary was hacked willnow makeposted clear. a dump of usernames and password hashes to About pastebin.com. © 2 01 1 TechCrunch INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) Jobs Network Staf f 08-78 SQL Code Injection Vulnerability SQL-Injection to infiltrate a SQL database with own SQL commands. Can be used to execute SQL statements with elevated privileges or to impersonate another user. Without direct database connection (e.g. web application) Injecting SQL via un-checked user input. Exploiting buffer overflows. Oracle standard packages have many buffer overflows. Output on attacker’s screen. With a direct database connection SQL Injection in built-in or user-defined procedures. Buffer overflows in built-in or user-defined procedures. Risk when a procedure is not defined with the AUTHID CURRENT_USER keyword (executes with the privileges of the owner INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-79 Hacking a Web Database Web-applications often construct a SQL-statement from separate strings. If a web-application does not thoroughly check the user’s input, in general every database on every operating system is vulnerable. Example: Consider the following SQL query in PHP $result=$conn->query('SELECT * FROM users WHERE username="'.$_POST['username'].'"'); The query selects all rows from the users table where the username is equal to the one put in the query string. Problem: quotes in $_POST['username'] not escaped & the string not validated Consider what would happen if we supply: " OR 1 OR username = " (a double-quote, followed by a textual " OR 1 OR username = " followed by another double-quote)…. Also, another line of SQL code can be added by adding a quote and a semicolon to the end so that the line… Many more problems possible… INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-80 Protecting a Web Database Be careful to check all parameters which can end up in such SQL statements! Never trust user provided data! Use dynamic SQL statements with explicit, type-checked parameters (bindValue() and bindParam() functions). Restrict the privileges of the user/role of the web application E.g. with Oracle: Revoke EXECUTE privilege on Oracle standard packages when not needed. Specially for the PUBLIC role. Patch, patch, patch ;-) Also: NEVER directly return database error messages Not very user-friendly AND it gives attackers hints INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-81 This Week’s Agenda Database Application Architectures Client-side DB Application Development Database-APIs: PDO and JDBC Database Application Design Principles Server-side DB Application Development Stored Procedures INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-82 Stored Procedures Run application logic within the database server Included as schema element (stored in DBMS) Invoked by the application Advantages: Central code-base for all applications Improved maintainability Additional abstraction layer (programmers do not need to know the schema) Reduced data transfer Less long-held locks DBMS-centric security and consistent logging/auditing (important!) Note: although named procedures, can also be functions INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-83 Stored Procedures Application (client) Call P Regular procedure DBMS (server) Network connection P table Intermediate results In/out arguments Call P Network connection P table Stored procedure INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-84 SQL/PSM Stored Procedures not only have full access to SQL All major database systems provide extensions of SQL to a simple, general purpose language SQL:1999 Standard: SQL/PSM PostgreSQL: PL/pgSQL Oracle: PL/SQL (syntax differs!!!) Extensions Local variables, loops, if-then-else conditions Example: CREATE PROCEDURE ShowNumberOfEnrolments SELECT uosCode, COUNT(*) FROM Enrolled GROUP BY uosCode Calling Stored Procedures: CALL statement Example: CALL ShowNumberOfEnrolments(); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-85 Procedure Declarations Procedure Declarations (with SQL/SPM) CREATE PROCEDURE name ( parameter1,…, parameterN ) local variable declarations procedure code; Stored Procedures can have parameters of a valid SQL type (parameter types must match) three different modes IN arguments to procedure OUT return values INOUT combination of IN and OUT CREATE PROCEDURE CountEnrolments( IN uos VARCHAR ) SELECT COUNT(*) FROM Enrolled WHERE uosCode = uos; CALL CountEnrolments (‘INFO2120’); INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-86 PostgreSQL: PL/pgSQL (cf. http://www.postgresql.org/docs/8.4/static/plpgsql.html) Extents SQL by programming language contructs Only knows functions! CREATE FUNCTION name RETURNS ... AS... Compound statements: BEGIN … END; SQL variables: DECLARE section variable-name sql-type; Assignments: variable := expression; IF statement: IF condition THEN … ELSE … END IF; Loop statements: FOR var IN range (WHILE cond ) LOOP … END LOOP; Return values: RETURN expression; Call statement: CALL procedure(parameters); Transactions: COMMIT; ROLLBACK; INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-87 Tip: CREATE OR REPLACE to avoid ‘name-already-used’ PL/pgSQL Example (cf. http://www.postgresql.org/docs/8.4/static/plpgsql-structure.html) PL/pgSQL procedure declaration CREATE OR REPLACE FUNCTION name ( parameter1, …, parameterN ) RETURNS sqlType AS $$ DECLARE optional variable sqlType; … BEGIN … Tip: final delimiter must match the one END; used after AS $$ LANGUAGE plpgsql; where parameterX is declared as (IN is default): [IN|OUT|IN OUT] name sqlType INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-88 PostgreSQL PL/pgSQL Example CREATE OR REPLACE FUNCTION RateStudent (studId INTEGER, uos VARCHAR) RETURNS CHAR AS $$ DECLARE grade CHAR; marks INTEGER; BEGIN SELECT SUM(marks) INTO marks FROM Assessment WHERE sid=$1 AND uosCode=$2; IF ( marks>84 ) THEN grade := ‘HD’; ELSIF ( marks>74 ) THEN grade := ‘D’; ELSIF ( marks>64 ) THEN grade := ‘CR’; ELSIF ( marks>50 ) THEN grade := ‘P’; ELSE grade := ‘F’; END IF; RAISE NOTICE 'Final grade is: %s', grade; RETURN grade; END; $$ LANGUAGE plpgsql; INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-89 Calling Stored Procedures from Clients Embedded SQL EXEC SQL BEGIN DECLARE SECTION char courseId(8); EXEC SQL END DECLARE SECTION EXEC SQL CALL CountEnrolments(:courseId); JDBC: CallableStatement cstmt = conn.prepareCall( “{call CountEnrolments(?)}”); cstmt.setString(1,courseId); cstmt.executeUpdate(); SQLJ #sql Iterator studnum(int count) #sql studnum = {CALL CountEnrolments(:courseId)} while ( studnum.next() ) { … } INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-90 Calling Stored Procedures from PDO Calling a Stored Procedure with parameters: (here: first IN, second an INOUT parameter) var $empname; var $empid = 42; $cstmt = $conn->prepare("CALL HighestPaidEmp(?,?)"); $cstmt->bindParam(1, $empid); $cstmt->bindParam(2, $empname, PDO::PARAM_STR|PDO::PARAM_INPUT_OUTPUT, 20); $cstmt->execute(); Specify as INOUT print $empname; Out strings require parameter with bitwisea max length or of type and inout flag The syntax for calling stored Functions is as follows: $stproc_stmt = $conn->prepare("?=CALL funcname(?,?,?)"); The first ? refers to the return value of the function and is also to be registered as an PDO::PARAM_INPUT_OUTPUT parameter. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-91 Calling Stored Procedures from JDBC Calling a Stored Procedure with parameters: (here: first IN, second an OUT parameter) CallableStatement cstmt = conn.prepareCall( “{call HighestPaidEmp(?,?)}”); cstmt.setInt(1, empid); cstmt.registeroutParameter(2, Types.VARCHAR); cstmt.executeUpdate(); String empname = cstmt.getString(2); The syntax for calling stored Functions is as follows: CallableStatement stproc_stmt = conn.prepareCall ("{ ? = call _funcname(?,?,?)}"); The first ? refers to the return value of the function and is also to be registered as an OUT parameter. INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-92 Externally Defined Stored Procedures Stored Procedures can also be defined using external code in a programming language Example: SQL/PSM CREATE PROCEDURE RankStudents ( IN number INT ) LANGUAGE JAVA EXTERNAL NAME ‘file:///c:/storedProcs/rank.jar’ Oracle PL/SQL Example: CREATE PROCEDURE RankStudents (number IN INT ) IS LANGUAGE JAVA NAME ‘file:///c:/storedProcs/rank.jar’ INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-93 Stored Procedure Engine in Oracle Pre-9i: Always interpreted execution Since 9i: also compiled native execution INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-94 Latest From Stored Procedures Virtual machines now ‘integrated’ with DBMS E.g. Java with Oracle .Net CLR with IBM, Oracle, and SQL Server PostgreSQL: Supports several scripting languages such as perl etc. MySQL: Working on Stored procedures in V5… alpha today But degree of integration differs heavily Oracle DBMS and Java VM: Two different processes Bad for performance because of context switches and data copying Similar with .Net integration in DB2 SQL Server 2005 & 2008: CLR tightly integrated into DBMS Should give better performance, but let’s see first… PostgreSQL: C-code dynamically linked to code But potential security thread… INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-95 CLR Integration in SQL Server 2008 Problem: CLR and database are two different runtime environments Both provide memory / thread management and synchronization Goals: Reliability, Scalability, Security, Performance SQL SERVER CLR SQL Server OS (memory, threads, synchronization) Also: UDTs, streaming functions, UDAs INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-96 DBMS Comparison DBMS Internal Stored Procedures External Stored Procedures C Java .NET CLR IBM DB2 SQL/PSM yes yes yes Oracle PL/SQL yes yes yes SQLServer T-Sql yes J# yes Sybase T-Sql (yes) yes no PostgreSQL PL/pgSQL; PL/Tcl; PL/Perl; PL/Python yes no no MySQL no no no since version 5; SQL/PSM syntax INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-97 Lessons Learned Same core issues for any db client-side development Data and type conversion: Host Variables NULL value semantic: Indicator variables and testing methods Impedance Mismatch: Cursor Concept Dynamic versus static SQL Database APIs You should in particular be able to write small PHP or JDBC programs DB Application Design Principles DAO Pattern; Error Handling; protection against SQL Injection Server-side database programming How to use stored procedures to run code inside a DBMS e.g. with PostgreSQL's pl/pgsql or with Oracle’s PL/SQL Modern database engines provide virtual machine environments to run external code near the data INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-98 References Kifer/Bernstein/Lewis (2nd edition) Chapter 8 Ramakrishnan/Gehrke (3rd edition - the ‘Cow’ book) Chapter 6 Ullman/Widom (3rd edition of ‘First Course in Database Systems’) Chapter 9 (covers Stored Procedures, ESQL, CLI, JDBC and PHP) Research Papers and Presentations: Acheson, et al.: “Hosting the .NET Runtime in Microsoft SQL Server”. SIGMOD 2004. E.M. Fayo: “Advanced SQL Injection in Oracle Databases”, Powerpoint presentation, February 2005. Database Documentation: PHP PDO extensions: http://www.php.net/manual/en/book.pdo.php The PostgreSQL Global Development Group: “PostgreSQL 8.2.4 Documentation”, 2009. Oracle Corporation: “Oracle 10.1 Database Concepts”,2003. MySQL website: http://www.mysql.com INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-99 Next Lecture (after Easter Break) Transaction Management Transaction Concept Serializability SQL Commands to Control Transactions Readings: Kifer/Bernstein/Lewis book, Chapter 18 or alternatively (if you prefer those books): Ramakrishnan/Gehrke (Cow book), Chapter 16 Ullman/Widom, Chapter 6.6 onwards INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-100