Download Grid-Anforderungen aus der Hochenergiephysik

Oracle @ DESY
DV-Seminar
26.01.2009
Harald Falkenberg / Christine Apfel
History
ongoing replacement of IBM/MVS main frame by Unix systems
 replacement needed for DB2 and AS
 migration of HERA documentation (magnet DB, vakuum DB),

H1, M, phonebook applications
reqirements on new database system:
 support multiple OS: IBM/MVS, Mac, Windows 3.x, VMS, Unix: AIX,
HP/UX,
 IRIX, SunOs, Solaris, Ultrix
 support of embedded C/C++, Fortran, Cobol libraries
 support of fill-in and reproting tools
 support of planed new applications:
 CAD/Cim database, account registry, date book,
 data analyses of HERA experiments (H1, Zeus)
Oracle start in 1993
decision to buy Oracle database licences due to:
 multiple user environment
 very broad OS support for server and client software
 support of developer and front-end tools: forms, graphics,
report
 support of programming languages (OCI)
In the end: start with oracle 7.1
History of oracle platforms
Oracle at DESY
System
Licences
SUN Enterprise 3000
4 CPUs 1 GB RAM
8 x 4 GB disc space
Solaris 5
DB 7.3.X
SGI Challenge
Irix 6.5
DB 7.1.X
Purchase 1993
100 concurrent DB users
10 developer user
OS: DOS-Windows, UNIX, MAC
SUN Cluster
2 E450, 4 GB RAM, 12 x 16 GB disc space
Solaris 7, CS 2.1
DB 8.1.7
Upgrade DB 9.2.0
Upgrade to Solaris 8, CS 2.2
Upgrading 1998
+ 50 concurrent DB users
RAC
3 x E 480
8 GB RAM
Solaris 10
DB 10.2.0
Oxygen 4000
Licence change 2002
150 concurrent DB users are
converted
6 CPU Licencen
Time
1994
1993
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
Oracle Usage and Applications

450 Accounts have Oracle Resource enabled in DESY-Registry

Oracle Applications (no complete list)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Aleph/ExLibris, AMS, CAD CIM-DB/W1,
DACHS/INTERFLEX, Datebook, DOOR, EDMS,
GINFO, GIS/FMS, GWO, div H1 Applications, HASYLAB,
HASYLAB/JB, HERA-AS, HERA-B,
IT-Info, KDS, div M-Applications,
M-Logbuch, Mail-PA/-CTL, Mail-Sweaper, Mail-Reporting
MKK-Cad/-Climate/-Logbook/-Power/-Water, MKS-AMS, MKS-Epicsora, MKS-Toolbox
MCS Webarchiv, SCICON, Self-Registration, EDUCourses, MPY/XFEL
PIP/Phonebook, PITZ, POL, div PT/HS Applications, PVSS, PUBDB,
Registry, Registry-NAF, RQS
NOC WF-Stat, INFEE, IT-Sec Apps
SPMS/JACOW, SYMPHA, TTF/XFEL-DB
WWW-Apps, Web-Server-Registry, div Zeus-Applications
a.s.o.
Statistics
the following administrative statistics are available in
an apex application
 user logons per day
 user average concurrent logon per day
 user logons per hour - concurrent
 database storage
Overview about oracle service
DatabaseAdministration
DatabaseSupport
SystemAdministration
Training /
Workshops
ApplicationsServerAdministration
Forms/Reports
Data-Backup
SystemBackup
Support in
Developertools
Migration
APEX
Migration
Documentation
Software
Supply
Documentation
Oracle RAC Cluster
CE 0
Interconnect
Private Switch
Private Switch
active
SUN E 480
4 CPUs, 8 GB RAM
Solaris 10
Oracle CS 10.2.0.X
Oracle DB 10.2.0.X
Oracle ASM 10.2.0.X
CE 1
standby
SUN E 480
4 CPUs, 8 GB RAM
Solaris 10
Oracle CS 10.2.0.X
Oracle DB 10.2.0.X
Oracle ASM 10.2.0.X
Data link
SUN E 480
4 CPUs, 8 GB RAM
Solaris 10
Oracle CS 10.2.0.X
Oracle DB 10.2.0.X
Oracle ASM 10.2.0.X
CE 2
CE 3
Storage
Oxygen 4000 (1,8 TB)
Public Switch
Public Switch
Public Network
Database Administration
 user/application
(Creation, Privileges, Resource management (Tablespace,
Quota, CPU, Memory usage))
 OS-, CS-, ASM-, Database monitoring and status reports
 AWR/workload repository and application monitoring
 performance findings and tuning advisors
 troubleshooting triggered by users and applications
 database services and SQL*NET/access to DBs
 ASM (automatic storage management)
 applying of patches [ rac software / db packages ]
goal:
keeping a shared resource usable for every user/application
Database/Data-Backup and
Recovery
Database-Administration

on-line Database Backup 2 times the week

archive logs and database backup saved on ASM backup
disk group

backup of ASM backup disk group in TSM

nightly export of user data (dumps)
User possibilities

undo retention of 4 hours (rollback)

recycle bin (dropped tables, not garantied)
Recovery options

recovering of database, tablespaces and datafiles

import of exported user data
Documentation
 providing of documentation on
http://www-it.desy.de -> services -> databases -> oracle
 first steps to use oracle at desy
 user environment in DB, description of tools
• sqlplus
• exp/imp
• sqlloader
 howtos to embed oracle access in c, java, perl, phython, php...
 howto use ODBC with MS Office products
 on-line books (oracle 10g documentation)
Trainings / Workshops
Training
 in house by desy oracle-service
 in house with Oracle trainers by request
 negotiation of fees (discount) for in-house and extern courses
Workshops
 organization of workshop events (also by user requests)
 inviting speakers from Oracle staff to actual topics
• Apex
• Forms/Reports – Migration → ADF
• BI-Publisher
• new SQL DB features
 DOAG in house talks
Support / Consultance
 e-mail address: [email protected]
 announcements via mailing list:
[email protected]
 consulting users:
• troubleshooting,
• data modelling,
• sql & pl/sql,
• performance & tuning,
• apex,
• j2ee/oc4j,
• bi-publisher,
• client software installation and configuration
database migration
Process of transforming data from a
database system to oracle database
system
Data
Data

ch
en r
kb e
or lop
W e
n ev
tio LD
ra
ig SQ
M ***
Data
the entire database schema include
trigger and stored procedures …
and data
database release change

Data
tion
Work
b
•
data is accurately transformed
•
data is complete
•
dependent applications are still
working
Data
***S
QLD
e
velo
Data
per
Data
Data
precheck for datatypes advisable
data verification to determine
whether
e n ch
Mig
io
ra t
o rk
nW
be
nc
h*
D
QL
**S
ev
elo
pe
Data
r
Data
Data
er / A
PEX

Migra
Data
velop

usefully performed programmatically ***more or less
Oracle Migration Workbench
SQL
De

Software Supply / Maintenance
Windows
 NetInstall products
• oracle client 10g (sqlplus, exp/imp, odbc/jdbc)
• sql developer
• jdeveloper
 forms/reports available by request
Linux (sl4 & sl5)
 oracle 10g client & sqldeveloper already available
 other tools available by request
Solaris 8 - 10
 oracle 10g client software
Processing of sql-statements
Current DB-parameter settings:
CURSOR_SHARING = EXACT
The database uses this parameter to control whether it will „auto-bind“
a SQL Statement.
Goals:
 scalability and throughput
 reducing load on the shared pool (sql area)
 parse once / execute many
 eliminating literal SQL
Processing chart
Statement Parser
Sqlstatement(s)
Set of query
blocks
SGA
Shared pool
CBO (Cost based optimizer)
Query transformation engine
Data dictionary cache
Estimator
Library cache
Plan generator
P-Code, Execution Plan
Query Execution
Processing of sql-statements
1. submit sql to server
2. determine meta data hash val (username, object-owner, session-parameter,...)
and hash of sql statement
3. determine if meta-hash and sql-hash exist in shared pool
if not you will get the following overhead:
- CBO determines execution plan (CPU, I/O, access statistics)
- parsing/compilation of sql (library cache)
- storing parsed sql in shared pool (I/O, shared pool consumption)
4. execution of parsed sql statement
conclusion:
sql statements with literals have always different hash-values and produce
overhead
using sql statements with bind variables benefit in avoiding overhead and
use less database resources and gain performance
Bind vs no Bind
With Bind Variables
Without Bind Variables
public void createStatement() throws SQLException {
rs = null;
public void createStatement() throws SQLException {
rs = null;
pstmt = null;
try {
// Create statement
pstmt = conn.prepareStatement("SELECT ? FROM dual");
stmt = null;
try {
// Get Connection
System.out.println("Start: " + new Date());
for (int i = 1; i < 100000; i++) {
stmt = conn.createStatement();
System.out.println("Start: " + new Date());
// Use bind variable for subsitution
pstmt.setString(1, String.valueOf(i));
for (int i = 0; i < 100000; i++) {
// Execute Statement
// Execute the query
rs = pstmt.executeQuery();
rs = stmt.executeQuery("SELECT " + i + " FROM dual");
}
System.out.println("End: " + new Date());
}
System.out.println("End: " + new Date());
} catch (SQLException se) {
System.out.println("SQL Exception:");
se.printStackTrace();
}
finally {
pstmt.close();
}
} catch (SQLException se) {
System.out.println("SQL Exception:");
se.printStackTrace();
}
finally {
// Close the result set, pstatement and the connection
stmt.close();
}
}
Die Ausführung der Klasse Bind liefert hingegen folgendes Resultat:
}
Die Ausführung dieser Klasse NoBind liefert folgendes Resultat:
Start: Mon Jan 26 09:18:50 CEST 2009
End: Mon Jan 26 09:19:41 CEST 2009
Start: Mon Jan 26 09:00:33 CEST 2009
End: Mon Jan 26 09:05:21 CEST 2009
APEX – Application Express
What is APEX
 a complete and integrated application development system
and runtime system for web applications
• application builder
• sql workshop with query builder
• utility area (data up-/download, schema ddl a.s.o.)
 development framework via normal internet browser
 applications are immediately available (after publishing the
URL)
 different authentication methods available incl. kerberos
 APEX is available with each oracle-Database incl. Oracle XE
 publicated as best successor for Forms & Reports
APEX Advantages
 intuitive user interface
 convenient assistants for application building
 interactive reports
 complete designed layout-template
 master detail forms and reports
 webservices per mouse-click available
 Flash-grafics
 printer friendly output (e.g. pdf, word)
 web compatible pl/sql editor
Note: APEX Workshop in June !!!
APEX @ DESY
 available at DESY since March 2008
 up to now 15 Workspaces with currently 28 Applications
(more or less complicated)
 combined with bi-publisher report engine to generate the
following output formats:
• html / xml
• pdf
• cvs
• excel / word
 flash grafics
Oracle Service
 contact: [email protected]
 in addition you can contact us any time
in case of
• planing of database applications
• data modelling
• evaluation of new database applications and
• any support questions