Download Java Database Connectivity JDBC Architecture JDBC - CS

Java Database Connectivity
• JDBC (Java Database Connectiveity) is an API
(Application Programming Interface),
– That is, a collection of classes and interfaces
JDBC (@PostgreSQL)
• JDBC is used for accessing (mainly) databases
from Java applications
• Information is transferred from relations to objects
and vice-versa
– databases optimized for searching/indexing
– objects optimized for engineering/flexibility
HUJI-CS
DBI 2008
JDBC Architecture
JDBC Architecture (cont.)
Network
Application
These are
Oracle
Driver
Java classes
2
JDBC
Driver
• Java code calls JDBC library
Oracle
• JDBC loads a driver
Java
Application
DB2
Driver
JDBC
• The driver talks to a particular database
• An application can work with several databases
by using all corresponding drivers
DB2
Postgres
Driver
We will
• Ideal: change database engines w/o changing
any application code (not always in practice)
use this one…
Postgres
HUJI-CS
DBI 2008
3
DBI 2008
Seven Steps
HUJI-CS
4
Registering the Driver
• Load the driver
• To use a specific driver, instantiate and
register it within the driver manager:
• Define the connection URL
Driver driver = new
org.postgresql.Driver();
DriverManager.registerDriver(driver);
• Establish the connection
• Create a Statement object
• Execute a query using the Statement
• Process the result
• Close the connection
DBI 2008
HUJI-CS
5
DBI 2008
HUJI-CS
6
A Modular Alternative
An Example
// A driver for imaginary1
Class.forName("ORG.img.imgSQL1.imaginary1Driver");
// A driver for imaginary2
Driver driver = new ORG.img.imgSQL2.imaginary2Driver();
DriverManager.registerDriver(driver);
//A driver for PostgreSQL
Class.forName("org.postgresql.Driver");
• We can register the driver indirectly using
Class.forName("org.postgresql.Driver");
• Class.forName loads the given class dynamically
• When the driver is loaded, it automatically
– creates an instance of itself
– registers this instance within DriverManager
• Hence, the driver class can be given as an
argument of the application
imaginary1 imaginary2
Postgres
Registered Drivers
HUJI-CS
DBI 2008
7
Connecting to the Database
HUJI-CS
DBI 2008
8
Connecting to the Database
Connection con =
• Every database is identified by a URL
DriverManager.getConnection("jdbc:imaginaryDB1");
• Given a URL, DriverManager looks for the
driver that can talk to the corresponding
database
acceptsURL("jdbc:imaginaryDB1")?
• DriverManager tries all registered drivers,
until a suitable one is found
a
r
imaginary1 imaginary2
• How is this done?
r
Postgres
Registered Drivers
DBI 2008
HUJI-CS
9
The URLs in HUJI-CS
DBI 2008
HUJI-CS
10
Interaction with the Database
In CS, a URL has the following structure:
• We use Statement objects in order to
jdbc:postgresql://dbserver/public?user=??
– Query the db
– Update the db (insert, update, create, drop, …)
• Three different interfaces are used:
Your login
Statement, PreparedStatement, CallableStatement
The machine
running PostgrSQL
• All are interfaces, hence cannot be instantiated
• They are created by the Connection
You can only access your own account!
DBI 2008
HUJI-CS
11
DBI 2008
HUJI-CS
12
Querying with Statement
Changing DB with Statement
String queryStr =
"SELECT * FROM Member " +
"WHERE Lower(Name) = 'harry potter'";
String deleteStr =
"DELETE FROM Member " +
"WHERE Lower(Name) = 'harry potter'";
Statement stmt = con.createStatement();
int delnum = stmt.executeUpdate(deleteStr);
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(queryStr);
• executeUpdate is for data manipulation: insert,
delete, update, create table, etc.
– Anything other than querying!
executeQuery returns a ResultSet object
representing the query result (discussed later…)
DBI 2008
HUJI-CS
• executeUpdate returns the number of rows modified
(or 0 for DDL commands)
13
About Prepared Statements
DBI 2008
HUJI-CS
14
Querying with PreparedStatement
• Parsed (compiled) by the DBMS only once
String q =
"SELECT * FROM Items " +
"WHERE Name = ? and Cost < ?";
• Values of some columns are set after compilation
PreparedStatement pstmt=con.prepareStatement(q);
• Prepared statements are used for queries that
are executed many times
• Instead of values, use ‘?’ and setType methods
pstmt.setString(1, "t-shirt");
pstmt.setInt(2, 1000);
• Hence, prepared statements can be thought of as
statements that contain placeholders to be
substituted later with actual values
DBI 2008
HUJI-CS
ResultSet rs = pstmt.executeQuery();
15
Updating with PreparedStatement
16
Are these the same? What do they do?
String val = "abc";
PreparedStatement pstmt =
con.prepareStatement("select * from R where A=?");
pstmt.setString(1, val);
ResultSet rs = pstmt.executeQuery();
PreparedStatement pstmt = con.prepareStatement(dq);
pstmt.setString(1, "t-shirt");
pstmt.setInt(2, 1000);
String val = "abc";
Statement stmt = con.createStatement( );
ResultSet rs =
stmt.executeQuery("select * from R where A=" + val);
int delnum = pstmt.executeUpdate();
HUJI-CS
HUJI-CS
Statement vs. PreparedStatement: Be Careful!
String dq =
"DELETE FROM Items " +
"WHERE Name = ? and Cost > ?";
DBI 2008
DBI 2008
17
DBI 2008
HUJI-CS
18
What can be Assigned to “?”
PreparedStatement and Security
• Suppose Google was implemented in JDBC
without a PreparedStatement. The main DB query
might have been implemented like this:
• Will this work?
PreparedStatement pstmt =
con.prepareStatement("select * from ?");
pstmt.setString(1, myFavoriteTableString);
Statement s;
s.executeQuery("SELECT URL,Title from Internet " +
"WHERE Content LIKE ‘%" + searchString + "%’");
• What would happen if a hacker searched for:
aaaaa’ UNION SELECT Company AS URL, CreditCardNum
AS Title FROM AdvertisingClients WHERE Company LIKE ‘
• No!!! “?” can only represent a column
value (to enable pre-compilation)
HUJI-CS
DBI 2008
• This technique is known as SQL Injection and is
the main reason for using PreparedStatements
19
DBI 2008
Timeout
HUJI-CS
20
ResultSet
• ResultSet objects provide access to the
tables generated as results of executing
Statement queries
• Use setQueryTimeOut(int seconds) of Statement
to set a timeout for the driver to wait for a
query to be completed
• If the operation is not completed in the
given time, an SQLException is thrown
• Only one ResultSet per Statement or
PreparedStatement can be open at a given
time!
• What is it good for?
• The table rows are retrieved in sequence
– A ResultSet maintains a cursor pointing to its
current row
– next() moves the cursor to the next row
HUJI-CS
DBI 2008
21
DBI 2008
ResultSet Methods
22
ResultSet Methods (cont’d)
• Type getType(int columnIndex)
• boolean next()
– Returns the given field as the given type
– Indices start at 1 and not 0!
– Add the column name as a comment if it is known!
– Activates the next row
– First call to next() activates the first row
– Returns false if there are no more rows
• Type getType(String columnName)
– Not all of the next calls actually involve the DB
– Same, but uses name of field
– Less efficient (but may not be your bottleneck anyway)
• void close()
• Examples: getString(5), getInt(“salary”), getTime(…),
getBoolean(…), ...
– Disposes of the ResultSet
– Allows to re-use the Statement that created it
• int findColumn(String columnName)
– Automatically called by most Statement methods
DBI 2008
HUJI-CS
HUJI-CS
– Looks up column index given column name
23
DBI 2008
HUJI-CS
24
ResultSet Example
Mapping Java Types to SQL Types
SQL Type
CHAR, VARCHAR, LONGVARCHAR
NUMERIC, DECIMAL
BIT
TINYINT
SMALLINT
INTEGER
BIGINT
REAL
FLOAT, DOUBLE
BINARY, VARBINARY, BYTEA
DATE
TIME
TIMESTAMP
Statement stmt = con.createStatement();
ResultSet rs = stmt.
executeQuery("select name,age from Employees");
// Print the result
while(rs.next()) {
System.out.print(rs.getString(1) + ":");
System.out.println(rs.getShort("age"));
}
DBI 2008
HUJI-CS
25
Java Type
String
java.math.BigDecimal
boolean
byte
short
int
long
float
double
byte[]
java.sql.Date
java.sql.Time
java.sql.Timestamp
HUJI-CS
DBI 2008
Null Values
26
Null Values
• In SQL, NULL means the field is empty
When inserting null values into placeholders
of a PreparedStatement:
• Not the same as 0 or “”!
– Use setNull(index, Types.sqlType) for primitive types
(e.g. INTEGER, REAL);
• In JDBC, you must explicitly ask if the lastread field was null
– For object types (e.g. STRING, DATE) you may
also use setType(index, null)
– ResultSet.wasNull(column)
• For example, getInt(column) will return 0 if
the value is either 0 or NULL!
DBI 2008
HUJI-CS
27
HUJI-CS
DBI 2008
Result-Set Meta-Data
28
Database Time
• Times in SQL are notoriously non-standard
A ResultSetMetaData is an object that can be
used to get information about the properties of
the columns in a ResultSet object
• Java defines three classes to help
• java.sql.Date
– year, month, day
An example: Write the columns of the result set
• java.sql.Time
ResultSetMetaData rsmd = rs.getMetaData();
int numcols = rsmd.getColumnCount();
– hours, minutes, seconds
• java.sql.Timestamp
for (int i = 1 ; i <= numcols; i++)
System.out.print(rsmd.getColumnLabel(i)+" ");
DBI 2008
HUJI-CS
– year, month, day, hours, minutes, seconds,
nanoseconds
– Usually use this one
29
DBI 2008
HUJI-CS
30
Cleaning Up After Yourself
Dealing With Exceptions
Remember: close Connections, Statements,
Prepared Statements and Result Sets
An SQLException is actually a list of exceptions
catch (SQLException e) {
while (e != null) {
System.out.println(e.getSQLState());
System.out.println(e.getMessage());
System.out.println(e.getErrorCode());
e = e.getNextException();
}}
con.close();
stmt.close();
pstmt.close();
rs.close()
DBI 2008
HUJI-CS
31
HUJI-CS
DBI 2008
32
General SQL Advise
• Take the time to carefully design your database
before you start coding – this will save you time
and frustration
• The same data can be organized in very different
ways – When designing your DB and when
deciding what your primary keys should be,
always think about typical use cases and deduce
from them which queries are most common and
which queries must run fastest
Transaction Management
• Add as many constraints as possible (NOT NULL,
UNIQUE, etc…) – this will help you debug prevent
data corruption even if you missed a bug
DBI 2008
HUJI-CS
33
Transactions and JDBC
An Example
Suppose that we want to transfer money from
bank account 13 to account 72:
• Transaction: more than one statement that must
all succeed (or all fail) together
PreparedStatement pstmt = con.prepareStatement("update
BankAccount set amount = amount + ? where accountId = ?");
– e.g., updating several tables due to customer purchase
• Failure− System must reverse all previous actions
pstmt.setInt(1,-100);
pstmt.setInt(2, 13);
pstmt.executeUpdate();
• Also can’t leave DB in inconsistent state halfway
through a transaction
• COMMIT = complete transaction
pstmt.setInt(1, 100);
pstmt.setInt(2, 72);
pstmt.executeUpdate();
• ROLLBACK = cancel all actions
DBI 2008
HUJI-CS
35
DBI 2008
What happens if
this update fails?
HUJI-CS
36
Transaction Lifetime
Committing a Transaction
How do we commit?
• Transactions are not opened and closed explicitly
• Explicitly invoking Connection.commit()
• A transaction starts on 1st (successful) command
– After a connection is established
• Implicitly
– After the previous transaction ends
– After every query execution, if AutoCommit is true
• A transaction ends when COMMIT or ROLLBACK
are applied
– When the user normally disconnects (i.e.,
appropriately closes the connection)
– Either explicitly or implicitly (see next 4 slides)
– In some DBs: After invoking a DDL command
(CREATE, DROP, RENAME, ALTER, …)
DBI 2008
HUJI-CS
37
DBI 2008
Automatic Commitment
• Rolling Back: Undoing any change to data
within the current transaction
• If AutoCommit is true (default), then every statement
is automatically committed
• The ROLLBACK command explicitly rolls
back (and ends) the current transaction
• If AutoCommit is false, then each statement is
added to an ongoing transaction
• ROLLBACK is implicitly applied when the
user abnormally disconnects (i.e., without
appropriately closing the connection)
• Change using setAutoCommit(boolean)
• If AutoCommit is false, need to explicitly commit or
rollback the transaction using Connection.commit()
and Connection.rollback()
HUJI-CS
39
DBI 2008
Fixed Example
HUJI-CS
HUJI-CS
40
Transaction Isolation
con.setAutoCommit(false);
try {
PreparedStatement pstmt =
con.prepareStatement("update BankAccount
set amount = amount + ?
where accountId = ?");
pstmt.setInt(1,-100); pstmt.setInt(2, 13); pstmt.executeUpdate();
pstmt.setInt(1, 100); pstmt.setInt(2, 72); pstmt.executeUpdate();
con.commit();
catch (SQLException e) { con.rollback(); …; }
DBI 2008
38
Rolling Back
• A Connection object has a boolean AutoCommit
DBI 2008
HUJI-CS
• How do different transactions interact?
• Does a running transaction see
uncommitted changes?
• Does it see committed changes?
• Don’t worry about this now – we will
discuss it in future lessons (see the
Appendix if you really can’t wait)
41
DBI 2008
HUJI-CS
42
LOBs: Large OBjects
• A database can store large pieces of data
– e.g., images or other files
• Sometimes, two distinguished types:
Managing Large Objects
– CLOB: Character large object (a large
number of characters)
In PosgreSQL-JDBC
– BLOB: Binary large object (a lot of bytes)
• Actual data is not stored in the table, only
a pointer (Object ID) to the data storage
HUJI-CS
DBI 2008
LOBs in PostgreSQL
44
Storing LOBs (1)
Create a new lob
Get an in-stream to Obj. 24
Get an out-stream Obj. 53
CREATE TABLE MyImages (
name VARCHAR(20) ,
content OID);
LOB
Manager
911
MyImages
• Given: A binary source (file, socket, etc.),
readable through an InputStream object
53
Name
Description content
image1
image2
Human face
Rabbit
53
113
24
113
• Goal: Store the content of the source in the
table MyImages
LOBs
HUJI-CS
DBI 2008
45
HUJI-CS
DBI 2008
Storing LOBs (2)
46
Storing LOBs (3)
1. Disable auto-commit (why???):
4. Insert a new row, with the LOB’s oid
con.setAutoCommit(false)
PreparedStatement pstmt = con.prepareStatement
("INSERT INTO MyImages VALUES (?, ?)");
2. Obtain the LOB manager:
• Downcast con into org.postgresql.PGConnection
pstmt.setString(1, imageName);
pstmt.setInt(2, oid);
pstmt.executeUpdate();
LargeObjectManager lobm = con.getLargeObjectAPI()
3. Create a new LOB
pstmt.close();
int oid = lobm.create(LargeObjectManager.READ |
LargeObjectManager.WRITE)
DBI 2008
HUJI-CS
47
DBI 2008
HUJI-CS
48
Storing LOBs (4)
Retrieving LOBs (1)
5. Fill the LOB with the input bytes
1. Obtain the LOB ID from the relevant row:
LargeObject obj = lobm.
open(oid, LargeObjectManager.WRITE);
PreparedStatement pstmt = con.prepareStatement
("select bytes from MyImages where name = ?");
pstmt.setString(1, filename);
int bytesRead = 0; byte[] data = new byte[4096];
while ((bytesRead = iStream.read(data)) >= 0)
obj.write(data,0,bytesRead);
ResultSet rs = pstmt.executeQuery(); rs.next();
int oid = rs.getInt(1);
6. Finalize
rs.close(); pstmt.close();
obj.close(); iStream.close(); con.commit();
DBI 2008
HUJI-CS
49
Retrieving LOBs (2)
3. Read the content of the large object (that acts as
an input/output stream)
• Downcast con into org.postgresql.PGConnection
while ((bytesRead = obj.read(data,0,data.length)) > 0)
oStream.write(data, 0, bytesRead);
LargeObjectManager lobm = con.getLargeObjectAPI();
LargeObject obj =
lobm.open(oid,LargeObjectManager.READ);
HUJI-CS
50
Retrieving LOBs (3)
2. Using oid, obtain a the corresponding LargeObject
instance from the LOB Manager
DBI 2008
HUJI-CS
DBI 2008
obj.close(); oStream.close();
51
HUJI-CS
DBI 2008
52
Entering the Command-Line UI
psql -hdbserver public
Appendix: PostgreSQL
Command-Line Interface
Server running
PostgreSQL
Database Name
Automatically enters your own part of
the database (based on your login)
Recommendation:
> echo ”alias psql ’psql –hdbserver public’” >> ~/.aliases
DBI 2008
HUJI-CS
54
A Screenshot
Additional Useful Commands
\dt
\d table-name
\dT
\i sql-file
\dl
\lo_export oid file
\lo_import file
\q
You can now write
SQL queries
DBI 2008
HUJI-CS
55
List all tables
Show table definition
List all types
Include SQL from file
List all large objects
Export a large object
Import a large object
Quit
HUJI-CS
DBI 2008
56
Transactions in Command Line
• By default, the command-line session runs
in AUTOCOMMIT mode
– That is, commit is implicitly applied after every
command
Appendix: Byte-Arrays
• To use transactions, you need to explicitly
specify the beginning and end of a
transaction by the commands BEGIN and
• At the end of the transaction, you either
apply COMMIT, ROLLBACK or END
DBI 2008
HUJI-CS
57
Representing Byte Arrays
Storing Byte Arrays
Store array b[] of bytes:
1. Prepare a row-inserting statement
PreparedStatement pstmt =con.prepareStatement
("INSERT INTO MyBStrings VALUES (?, ?)");
Use the SQL type BYTEA to hold byte arrays
2. Set the statement values:
CREATE TABLE MyBStrings (
name VARCHAR(20) ,
content BYTEA);
pstmt.setString(1, desiredName);
pstmt.setBytes(2, b);
3. Execute and close
pstmt.executeUpdate();
pstmt.close();
(works with PostgreSQL, may not work with other DBMSs)
DBI 2008
HUJI-CS
59
DBI 2008
HUJI-CS
60
Retrieving Byte Arrays
Retrieve bytes with the name “myBytes”:
1.
Apply a selection query
PreparedStatement pstmt =con.prepareStatement
(“SELECT content from MyBStrings where name=?");
pstmt.setString(1, "myBytes");
ResultSet rs = ps.executeQuery();
2.
Appendix: Transaction
Isolation
Obtain the byte-array from the result
rs.next();
byte[] theBytes = rs.getBytes(1);
3.
Close resources
rs.close(); pstmt.close();
DBI 2008
HUJI-CS
61
Transaction Isolation
Transaction Isolation Levels
• The isolation level determines the capabilities
of a transaction to read/write data that is
accessed by other transactions
• How do different transactions interact?
• In PostgreSQL, two levels of isolation:
• Does a running transaction see
uncommitted changes?
1. READ COMMITTED (default)
2. SERIALIZABLE
• Does it see committed changes?
• Each transaction determines its isolation level
– Connection.setTransactionIsolation(int level)
DBI 2008
HUJI-CS
63
DBI 2008
Isolation Level in the Command-Line UI
READ COMMITED & SERIALIZABLE
• SERIALIZABLE: During the whole transaction,
statements read only the changes that were
committed by the time the transaction begun (and
the changes made by the transaction itself)
Changing to serializable:
SET SESSION CHARACTERISTICS AS TRANSACTION
ISOLATION LEVEL SERIALIZABLE;
HUJI-CS
64
• In principle, a query in PostgreSQL never reads
uncommitted (i.e. dirty) data
Changing to read committed:
SET SESSION CHARACTERISTICS AS TRANSACTION
ISOLATION LEVEL READ COMMITTED;
DBI 2008
HUJI-CS
• READ COMMITTED: A statement reads the data
that was committed by the time the statement (not
the transaction) begun
65
DBI 2008
HUJI-CS
66
Update: Row-Level Locking
Some Definitions
• PostgreSQL disables one from updating a row
that is updated by an uncommitted transaction
• Dirty reads: A transaction reads data that is
written by another, uncommitted transaction
– The second updating transaction is blocked until the
first one commits or rolls back
• Non-repeatable reads: A transaction rereads
• If 2nd update is within READ COMMITTED, then
2nd update is done after 1st transaction commits
or rolls back
data it previously read and finds that a committed
transaction has modified or deleted that data
• Phantom reads: A transaction re-executes a
• If 2nd update is within SERIALIZABLE, then 2nd
update fails if 1st transaction commits; otherwise,
it succeeds
HUJI-CS
DBI 2008
query returning a set of rows satisfying a search
condition and finds that a committed transaction
inserted additional rows that satisfy the condition
67
HUJI-CS
DBI 2008
68
What Happens Here (1)?
READ COMMITED vs. SERIALIZABLE
1. CREATE TABLE pairs (x INTEGER, y INTEGER);
READ COMMITED
SERIALIZABLE
Impossible
Impossible
Possible
Impossible
Possible
Impossible
Dirty Reads
Non-repeatable
Reads
Phantom
Reads
2. select * from pairs
3. insert into pairs values(1,1)
4. select * from pairs
5. insert into pairs values(1,2)
6. select * from pairs
7. select * from pairs
8. COMMIT
9. COMMIT
T.1: R. COMMITTED
T.2: SERIALIZABLE
HUJI-CS
DBI 2008
69
HUJI-CS
DBI 2008
70
What Happens Here (2)?
What Happens Here (3)?
1. CREATE TABLE pairs (x INTEGER, y INTEGER);
1. CREATE TABLE pairs (x INTEGER, y INTEGER);
2. insert into pairs values(1,1)
2. insert into pairs values(1,1)
3. select * from pairs
3. select * from pairs
4. COMMIT
5. select * from pairs
4. COMMIT
5. select * from pairs
6. select * from pairs
7. insert into pairs values(1,2)
8. COMMIT
9. select * from pairs
10. COMMIT
6. select * from pairs
7. insert into pairs values(1,2)
8. COMMIT
9. select * from pairs
10. COMMIT
T.2: SERIALIZABLE
T.1: SERIALIZABLE Is it equivalent to any truly serial
T.1: R. COMMITTED
DBI 2008
T.2: SERIALIZABLE
execution of the transactions?
HUJI-CS
71
DBI 2008
HUJI-CS
72
What Happens Here (4)?
What Happens Here (5)?
1. CREATE TABLE pairs (x INTEGER, y INTEGER);
1. CREATE TABLE pairs (x INTEGER, y INTEGER);
2. insert into pairs values(1,1)
3. COMMIT
2. insert into pairs values(1,1)
3. COMMIT
4. update pairs set y=2
where x=1
4. update pairs set y=2
where x=1
5. update pairs set y=3
where x=1
5. update pairs set y=3
where x=1
6. select * from pairs
7. COMMIT
8. select * from pairs
9. COMMIT
6. select * from pairs
7. COMMIT
8. select * from pairs
9. COMMIT
T.2: SERIALIZABLE
T.1: SERIALIZABLE
DBI 2008
T.2: SERIALIZABLE
T.1: R. COMMITTED
HUJI-CS
73
DBI 2008
HUJI-CS
74
Changing Your Password
Suppose that snoopy wants to change his
password from “snoopy” to “snoopass”:
Appendix: Changing
Passwords
ALTER USER snoopy WITH PASSWORD 'snoopass';
DBI 2008
HUJI-CS
76