* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download 点击这里下载
Survey
Document related concepts
Relational algebra wikipedia , lookup
Concurrency control wikipedia , lookup
Tandem Computers wikipedia , lookup
Oracle Database wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Microsoft Access wikipedia , lookup
Ingres (database) wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Clusterpoint wikipedia , lookup
Database model wikipedia , lookup
Relational model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Transcript
Application Development on Power System (IBM i) Application Development on Power System (IBM i) School of Software Engineering Tongji University HUANGJie Syllabus No Subject Hours Abstract Instructor Server, Power System and Power System IBM i Huangjie iOS fundamental 2 4 i OS & Basic Operation Huangjie 3 File system & object management 4 Object management concept and integrated file system Huangjie 4 Message management 2 Message & Its management Huangjie 5 Work management 4 Concepts of job, subsystem and library, job routing Huangjie 6 Administration ABC 2 Security, Storage & Restore Management Huangjie 7 Application Development 6 ILE development enviroment , RPG programming Huangjie 8 Database Development 4 Database on IBM i Huangjie 9 IBM i & SOA 2 SOA approach on IBM i Huangjie 10 Case Study 2 Csae Study Huangjie 11 Examination 2 Examination 1 Overview of IBM i 2 2017/5/22 Huangjie 2 Unit 3 IBM i Application Development Chapter 8 Database Application Development 2017/5/22 3 Outline • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心元素 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator 基本操作 2017/5/22 4 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心元素 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator 基本操作 2017/5/22 5 DB2 Family (ZOS, LUW, IOS) Marketshare Source: Gartner Press Release (5/30/05) 2017/5/22 * Estimate 6 History of DB2 for i System/38 AS/400 iSeries Initial SQE Ship 1980 Integrated Relational Database 2017/5/22 1988 SQL Interface SQL/400 1994 DB2 for AS/400 System i Today 1999 2001 2006 DB2 DB2 Universal Universal Database Database for AS/400 for iSeries DB2 for i5/OS 7 DB2 on IBM i 框架 OmniFind Text Search Server for DB2 for i OmniFind Text Search Server Clients IBM i OS OS/400 i5/OS Commands DB2 Query Mgr SQL Server Data Management Transaction Management ODBC Storage Management Resource Management DRDA/DDM Server Message s JDBC Server SN A SQL Security Message Management Index Management . . . TCP/I P SQL CLI SQLJ Applications Other OS/400 Primitives Native DRDA/DD Server Query/400 CLI Client Application DRDA Client Application DB2 WebQuery for IBM i ODBC Application Client JDBC Client Application Or Applet Application ADO Client Application OLE DB Application Client DDM Client IBM i DB2 Storage Engine for MySQL Application Ops Navigator PHP Application DB2 APIs Client Access RPG Application Writes to MySQL Data stored in 8 DB2 for i Accesses Data DB2 i Key Differences • Not a separate product – integrated into IBM i • i operating system uses DB2 for i and vice versa • Not a separate charge, but a few priced options – – – – • • • • • DB2 Multisystem DB2 Symmetric Multiprocessing Query Manager and SQL Development Kit WebQuery Native and SQL Interfaces Excellent interoperability and national language support Multiple partitions and operating systems Easier to use and maintain Management GUI is IBM i Navigator & Director 2017/5/22 2017/5/22 9 9 DB2 for i Interfaces Native Interfaces SQL Interfaces JDBC Cobol RPG C or C++ … START READ … … SETLL READ … … Ropen Rread … … START READ … … SETLL READ … C or C++ … Ropen Rread … 2017/5/22 RPG C or C++ Embedded SQL Cobol SQL Table or View or DDS Physical or Logical File Distributed Data Management (DDM RPG Cobol … SQLExecDirect … SQL FETCH SQLExecDirect … … SQLFETCH SQLExecDirect … SQLFETCH … 123 Fish, Joe 124 Olson, Ole 125 Smith, Sally 126 Johnson, John 127 Smith, Jim Cobol CLI Toolbox JDBC Driver Native JDBC Driver Host Server (ZDA) System i Access Optimized APIs System i Access Drivers JDBC ODBC .NET ADO 555 501 555 501 555 20000 20000 32000 38000 30000 Cobol RPG C or C++ … QSQPRCED… … QSQPRCED … Distributed Relational Database (DRDA) JDBC ODBC .NET ADO C or C++ … EXEC SQL … FETCH EXEC SQL … … FETCH EXEC SQL … FETCH … 001 501 555 001 555 DB2 Connect Drivers RPG Cobol … QSQPRCED … QSQPRCED API Extended Dynamic Remote SQL (XDA) RPG C or C++ … EXEC SQL … FETCH EXEC SQL … … FETCH EXEC SQL … FETCH … JDE SAP … 10 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心元素 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator基本操作 2017/5/22 11 Database Objects – Terminology SQL 2017/5/22 IBM i schema/collection library table physical file view logical file index keyed logical file row record column field log journal 12 Create a Physical /Logical File with DDS STRPDM PFILE1 QDDSSRC DESCRIPTION DESCRIPTION STRSEU PFILE1 …… LFILE1 …… CRTPF DATA MEMBER CRTLF LFILE1 DESCRIPTION MEMBER 2017/5/22 2017/5/22 13 13 Create a Physical File with DDS CRTPF FILE(LIBNAME/PFNAME) SRCFILE(DDSLIB/DDSFILE) 2017/5/22 2017/5/22 14 14 Create a Logical File with DDS CRTLF FILE(LIBNAME/LFNAME) SRCFILE(DDSLIB/DDSFILE) 2017/5/22 2017/5/22 15 15 SQL Components Data Definition CREATE DROP ALTER Security GRANT SQL REVOKE Structured Query Language DYNAMIC Data Manipulation SELECT INSERT UPDATE DELETE 2017/5/22 2017/5/22 MISCELLANEOUS CONNECT DECLARE DESCRIBE EXECUTE PREPARE 16 16 Create Table/View with SQL CREATE TABLE TB_WAREHOUSE ( WAREHOUSE_ID CHAR(10) NOT NULL WITH DEFAULT , WAREHOUSE_NAME CHAR(10) NOT NULL WITH DEFAULT , WAREHOUSE_STREET_1 VARCHAR(256) NOT NULL WITH DEFAULT , WAREHOUSE_CITY VARCHAR(40) NOT NULL WITH DEFAULT , WAREHOUSE_STATE CHAR(2) NOT NULL WITH DEFAULT , WAREHOUSE_ZIP CHAR(9) NOT NULL WITH DEFAULT '000000000', WAREHOUSE_TAX DECIMAL(7 , 4) NOT NULL WITH DEFAULT , WAREHOUSE_YTD US_DOLLAR NOT NULL WITH DEFAULT , WAREHOUSE_STREET_2 VARCHAR(40) WITH DEFAULT NULL, WAREHOUSE_CITYID INTEGER WITH DEFAULT NULL, LATITUDE DOUBLE NOT NULL WITH DEFAULT 0, LONGITUDE DOUBLE NOT NULL WITH DEFAULT 0 ); CREATE VIEW VW_WAREHOUSE (WAREHOUSE_ID, WAREHOUSE_NAME, WAREHOUSE_STREET_1, WAREHOUSE_STREET_2, WAREHOUSE_CITY, WAREHOUSE_STATE, WAREHOUSE_ZIP, WAREHOUSE_TAX, WAREHOUSE_YTD) AS SELECT WAREHOUSE_ID, WAREHOUSE_NAME, WAREHOUSE_STREET_1, WAREHOUSE_STREET_2, WAREHOUSE_CITY, WAREHOUSE_STATE, WAREHOUSE_ZIP, WAREHOUSE_TAX, WAREHOUSE_YTD FROM TB_WAREHOUSE WHERE WAREHOUSE_YTD > US_DOLLAR(100000); 2017/5/22 17 2017/5/22 17 Create Constraints with SQL 使用数据库约束条件的优势 – 代码重用/模块化 – 更好的数据完整性 – 更好的查询性能 - SQE 优化器可以利用约束条件产生最优的执行计划 约束条件的类型 – – 主键 & 唯一键 参照完整性约束 • 强制实现父/子约束关系 – 检查完整性 • 确保列中只能存放有效的值 CREATE TABLE orders( ordnum INTEGER PRIMARY KEY, ordqty INTEGER CHECK(ordqty>0 AND ordqty<1000), ordamt DECIMAL(7,2), part_id CHAR(4), CONSTRAINT ordpart FOREIGN KEY(part_id) REFERENCES parts(PartID) ON DELETE RESTRICT ON UPDATE RESTRICT ) 2017/5/22 18 Partitioned table • WHAT – A partitioned table is a table whose data is contained in one or more local partitions (members). – Range partitioning – Hash partitioning – Partitioning key • WHY – With partitioning V.S. Without partitioning – Partitioning can also enhance the performance, recoverability, and manageability of your database. – Most of the CL commands that are used for members are also valid for each partition of a partitioned table. • WHEN – V5R3, V7R1, V7R2, etc. 2017/5/22 19 Creation of partitioned tables • E.g. to partition table PAYROLL in library PRODLIB with partitioning key EMPNUM into four partitions, use the following code: • • • • • • • CREATE TABLE PRODLIB.PAYROLL (EMPNUM INT, FIRSTNAME CHAR(15), LASTNAME CHAR(15), SALARY INT) PARTITION BY HASH(EMPNUM) INTO 4 PARTITIONS • Or, to partition PAYROLL by range, use the following code: • • • • • • • • • CREATE TABLE PRODLIB.PAYROLL (EMPNUM INT, FIRSTNAME CHAR(15), LASTNAME CHAR(15), SALARY INT) PARTITION BY RANGE(EMPNUM) (STARTING FROM (MINVALUE) ENDING AT (500) INCLUSIVE, STARTING FROM (501) ENDING AT (1000) INCLUSIVE, STARTING FROM (1001) ENDING AT (MAXVALUE) 2017/5/22 20 Why MQT • To enhance query performance by using pre-calculated result set instead of re-computing them • Though MQT, user created summary tables may be known by DB2 optimizer, thus can be used to enhance query performance • Use of MQT is transparent to users 2017/5/22 21 MQT Example MQT1: CREATE TABLE MQT1 AS ( SELECT A.aid, T.fpgid, A.status, T.qty, T.price, T.disc, T.qty*T.price as value FROM Trans T, Acct A WHERE T.faid=A.aid AND T.disc>0.1) DATA INITIALLY DEFERRED REFRESH DEFERRED ENABLE QUERY OPTIMIZATION MAINTAINED BY USER Q1: SELECT A.aid, A.status, T.qty*T.price *(1-T.disc) as amt FROM Trans T, PGroup P, Acct A WHERE P.pgid=T.fpgid and T.faid=A.aid AND T.price> 100 AND T.disc>0.1 AND P.pgname>’TV Resulting New Query: SELECT A1.aid, A1.status, A1.value *(1-A1.disc) as amt FROM MQT1 A1, PGroup P WHERE P.pgid=A1.fpgid AND A1.price> 100 AND P.pgname>’TV 2017/5/22 22 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心特征 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator基本操作 2017/5/22 23 Stored Procedure An SQL "Program" that contains: • HLL Program with or without embedded SQL • SQL Language • Stored Procedure Created by CREATE PROCEDURE • Can pass parameters • Can return parameters, result set • Widely used in Client / Server applications to reduce traffic OR 2017/5/22 24 Stored Procedure STORED PROCEDURE CREATE PROCEDURE SPName (IN DataType Var1, Out ….. ) LANGUAGE SQL BEGIN INSERT INTO TableName VALUES(Var1); …… END 存储过程:将常用的或很复杂的工作,预先用SQL语句写好并用一个指定 的名称存储起来,那么以后要叫数据库提供与已定义好的存储过程的功能 相同的服务时,只需调用execute,即可自动完成命令。 2017/5/22 25 Trigger Triggers: Application-independent •Enforce business rules •Written once; used by many •Enforce data validation and audit trail Activated by database manager when •Preserve data consistency operations performed on database Triggers versus Stored Procedures •'Fired' by specified database •Triggers invoked by database events operations and take action written •Stored procedures invoked by application in trigger CALL •Interface-independent When a new order is inserted or updated, a trigger is fired: • Trigger retrieves information about order and customer • A confirmation fax is automatically sent 2017/5/22 26 Trigger Example Client Application Communication Line Data Flow Database Server with No Triggers TABLE1 UPDATE TABLE1 . . SELECT TABLE2. . . TABLE2 UPDATE TABLE2 . TABLE3 INSERT TABLE3 . . Client Application Communication Line Data Flow UPDATE TABLE1. . . Database Server With Triggers 当在一个物理文件上加载 触发器时,必须定义三个 属性。第一是定义引起触 发器运行的事件,这类事 件可以是在文件中插入、 更新或删除一条记录;第 二是定义触发器在什么时 候,即事件之前还是之后 启动;第三是定义要运行 的特定的程序,触发程序 是用户提供的基于任何 IBM i高级语言的程序。 TABLE1 TABLE2 TABLE3 SELECT TABLE2 UPDATE TABLE2 INSERT TABLE3 ..... Trigger Program 2017/5/22 27 Trigger Mode: DB2ROW versus DB2SQL Row Level、Statement Level、Column Level 2017/5/22 28 将业务逻辑移入数据库—Trigger 用户可以使用触发器对新增加的数据或者被修改的数据进行业务处理 – – – DB2保证触发器总是会被执行 触发器执行与用户所使用的接口无关 可以使用触发器在数据写入数据库前对其进行转换 DB2 for i 的触发器的支持 – – 前触发/后触发: 插入,更新和删除 (最多 300 个触发器可以被一次操作触发) SQL 和 外部(ADDPFTRG) 触发器 • 列级别,语句级别,以及Instead Of trigger只能使用SQL触发器 CREATE TRIGGER audit_salary AFTER UPDATE ON employee(salary) REFERENCING NEW AS n REFERENCING OLD AS o FOR EACH ROW WHEN (n.salary - o.salary >= 5000) INSERT INTO audit VALUES(n.empno, n.deptno, n.salary,current timestamp) 2017/5/22 29 The Need for Large Objects (LOBs) • Provide support for Web-based multimedia applications • Provide a built-in data type that can store a large amount of data • There are three different types of data: – – – 2017/5/22 BLOB (Binary Large Object): For scanned documents, digital images, and so forth CLOB (Character Large Object): Large character string data type DBCLOB (Double Byte Character Large Object): Large double-byte character string data type 30 User-Defined Types – Example INTEGER DOUBLE TIMESTAMP VARCHAR CLOB SMALLINT * / + SUM AVG DECIMAL DECIMAL CHAR SUBSTR POSSTR CREATE DISTINCT TYPE US_DOLLAR AS DECIMAL(11,2) WITH COMPARISONS CREATE DISTINCT TYPE EURO AS DECIMAL(11,2) WITH COMPARISONS 2017/5/22 BLOB DATE DAYS MONTH US_DOLLAR DECIMAL US_DOLLAR EURO DECIMAL EURO TIME MINUTE HOUR CREATE DISTINCT TYPE us_dollars AS DECIMAL(11,2); CREATE DISTINCT TYPE can_dollars AS DECIMAL(11,2); CREATE TABLE products ( prod_id CHAR(5), prod_desc VARCHAR(50), prod_weight DECIMAL(11,2), usa_price US_DOLLARS, can_price CAN_DOLLARS, product_image BLOB(1M) ) 31 User-Defined Functions – Examples FIND_PICTURE( ,SLIDE_SHOW) Large Object Search 15, 78, 54, 67, 67, 94, 99, 53, 47, 87, 72, ARRAY_AVERAGE(CLASSROOM_MARKS) 34, 63, 54, 59, 82 Type Cast Functions: Built-in Functions: SQRT() TAN() 2017/5/22 EXP() LOG() SIN() COS() RAND() POWER() MONEY() DOLLAR() 32 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心特征 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator 基本操作 2017/5/22 33 IBM i as “DB Server” IBM i IBM i JDBC CLI* Embedded DRDA Native/DDM AIX JDBC DRDA iSeries Toolbox JDBC DB2 Connect DB2 Information Integrator Linux JDBC ODBC DRDA iSeries Toolbox JDBC DB2 Connect iSeries ODBC DB2 for IBM i Windows JDBC ODBC OLE DB .NET DRDA iSeries Access DB2 Connect DB2 Information Integrator * OS/400 PASE only supports CLI DB2 for IBM i有多种编程访问接口,既有基于主机方式的接口,又有基于客户机/服务器方式的访问接口 2017/5/22 34 DB2 for i Interfaces Native Interfaces SQL Interfaces JDBC Cobol RPG C or C++ … START READ … … SETLL READ … … Ropen Rread … … START READ … … SETLL READ … C or C++ … Ropen Rread … 2017/5/22 RPG C or C++ Embedded SQL Cobol SQL Table or View or DDS Physical or Logical File Distributed Data Management (DDM RPG Cobol … SQLExecDirect … SQL FETCH SQLExecDirect … … SQLFETCH SQLExecDirect … SQLFETCH … 123 Fish, Joe 124 Olson, Ole 125 Smith, Sally 126 Johnson, John 127 Smith, Jim Cobol CLI Toolbox JDBC Driver Native JDBC Driver Host Server (ZDA) System i Access Optimized APIs System i Access Drivers JDBC ODBC .NET ADO 555 501 555 501 555 20000 20000 32000 38000 30000 Cobol RPG C or C++ … QSQPRCED… … QSQPRCED … Distributed Relational Database (DRDA) JDBC ODBC .NET ADO C or C++ … EXEC SQL … FETCH EXEC SQL … … FETCH EXEC SQL … FETCH … 001 501 555 001 555 DB2 Connect Drivers RPG Cobol … QSQPRCED … QSQPRCED API Extended Dynamic Remote SQL (XDA) RPG C or C++ … EXEC SQL … FETCH EXEC SQL … … FETCH EXEC SQL … FETCH … JDE SAP … 35 Native数据访问和SQL数据访问 • RPG 行层级访问 – 很好的单记录 / 顺序读取的 IO 性 能 – 当处理复杂业务逻辑时较难编程/ 维护 – 不易扩展 • SQL 36 – IBM最先进的查询优化器继续带 来更好的 性能 – 对复杂 业务需求 的快速交付 – 优秀的扩展性 – DB2 for i 数据存取的 发展方向 – 商业标准 的数据处理语言 – 开放性 带来更好的可移植性以及 更好的人才储备 – 增强的 数据完整性 Native向SQL迁移的主要考虑因素 传统模式 SQL 对象定义 DDS SQL DDL 数据建模 物理文件/逻辑文 件 表/视图/索引 数据访问 RPG/COBOL 行 级别访问 嵌入式/动态SQL 语句表级别访问 RPG程序员 数据库优化器 查询计划决定 者 Native记录级别访问和SQL的典型应用场景 推荐使用SQL开发新的应用,但是行级别访问不会消亡 选择SQL或者行级别访问依赖于应用场景以及应用架构 • SQL的优势 – – – – 以集合方式处理数据 大量的SELECT操作 复杂的业务逻辑 SQL提供强大的内建函数 与功能 记录级别访问的优势 – 在线交易类型的处理 – 对单条数据处理,读/写/ 更新 – 内建的对称多处理保证业 务可扩展性 2017/5/22 37 传统记录级别访问 38 以数据为核心的SQL编程 39 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心特征 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator基本操作 2017/5/22 40 Five Factors of DB Performance • Workload – OLTP, OLAP, ad-hoc queries, system commands • Can fluctuate drastically • Throughput – Overall capability to process data • CPU speed, I/O speed, parallel capabilities • Resources – Hardware- and software tools • Database kernel, disk storage, RAM, cache, microcode • Optimization – Query optimization is primary accomplished internal – SQL formulations, index strategy and database parameter • Contention – Condition, where two or more components are attempting to use a single resource in a conflict way – As contention increases, throughput decreases 2017/5/22 41 Various Factors of Optimization Server attributes Server configuration Version/Release/Modification Level Server performance SMP Job, Query attributes The Plan Table sizes, number of rows SQL Request Static Dynamic Extended Dynamic Interfaces 2017/5/22 Database design Views and Indexes (Radix, EVI) Work management 42 SQL Optimizer • Why? – Improved Database performance • Identify technique used to implement query • Selects most efficient technique • Access Plan – Internal structure defining method to process SQL statement – Created during SQL parse, syntax check • Optimal access method selection based on – Implementation cost • SQL Statement • Current state of the database –SQLIndexes/File Sizes Statement Optimization Bind Access Plan Database 2017/5/22 43 SQL Packages • IBM i Extended Dynamic SQL Support – – – – – Control structures used to execute SQL statements Access Plans stored in *SQLPKG object Improves Performance IBM i CL commands : CRTSQLPKG, DLTSQLPKG Dynamic re-optimization • File size • New/Deleted indexes First time SQL Statement Optimization Database Subsequent SQL Statement Access Plan Bind Access Plan Validate SQL Package Open Data Path (ODP) Database 2017/5/22 44 Parallelism 2017/5/22 45 Indexing • Radix Index • Encoded Vector Index • When will be indexes created? 2017/5/22 46 Index Strategy • DB2 for i has two types of indexing technologies – Radix Index – Encoded Vector Index • Each type of index has specific uses and advantages • Respective indexing technologies complement each other • Indexes are used for both statistics and implementation choices • Indexes provide RRN/Rids and/or data (Index Only Access Plans) • Indexes are scanned or probed – Probe can only occur on contiguous, leading key columns • Indexes support both relationships between tables (Referential Integrity) as well as support local selection, joining, grouping and ordering 2017/5/22 47 Radix Index Key values are compressed Common patterns stored once Unique portion stored in "leaf" pages Positive impact on size and depth of the "tree" Algorithm used to find values Binary search Very efficient process to find a unique value or small range of values Modified to fit the data structure Used to materialize a bitmap or relative record number (RRN) Maintenance Index data is automatically spread across all available disk units Tree is automatically "rebalanced" to maintain an efficient structure No "index reorganization" required 2017/5/22 48 Radix Index ROOT Database Table 001 ARKANSAS 002 MISSISSIPPI 003 MISSOURI 004 IOWA 005 ARIZONA … … Test Node ADVANTAGES: Very fast access to a single key value Also fast for small, selected range of key values (low selectivity) Provides order 2017/5/22 IOWA 004 AR IZONA 005 MISS ISSIPPI 002 OURI 003 KANSAS 001 DISADVANTAGES: Table rows retrieved in order of key values (not physical order) which equates to random I/O’s No way to predict which physical index pages are next when traversing the index for large number of key values 49 Encoded Vector Index (EVI) Index object for delivering fast data access in analytical query and reporting environments Advanced technology from IBM Research, that is a variation on bitmap indexing Complement to radix index (keyed logical file or SQL index) Easy to access data statistics improve query optimizer decision making Used to materialize a bitmap or relative record number (RRN) list Easy to maintain 2017/5/22 2017/5/22 50 50 Encoded Vector Index (EVI) Object type is File, subtype is LF EVI is composed of two parts... SYMBOL TABLE: Key Value Alabama Alaska Arizona California Colorado ... Wisconsin Wyoming VECTOR: Code Count 1 2 3 4 5 ... 49 50 1000 450 5000 10000 6500 ... 340 2760 Only Codes 1 17 5 9 2 7 49 49 5 ... Row 1 2 3 4 5 6 7 8 9 ... Symbol table contains information for each distinct key value. Each key value is assigned a unique code Code is 1, 2, or 4 bytes - depending on number of distinct key values Rather than a bit array for each distinct key value, the index has one array of codes (a.k.a., the Vector) 2017/5/22 2017/5/22 51 51 Encoded Vector Indexing New type of index that can significantly improve performance, especially for star schema 10% to 30% faster index builds 1/3 to 1/16 the size 1/2 the time for index scans 350 GB Table Query 1/3 the time for bit map generation 4 Table Join (Star Schema) Symbol Table Arizona Arkansas ...... Virginia Wyoming Code First Row 1 1 2 5 37 38 1222 7 Last Row 80005 99760 30111 83000 Count 300 5000 7300 Time (Minutes) Key Value 350 340 2760 13 12 28 2 17 250 200 150 100 Vector 1 4.8 Hours 38 2 26 33 29 Minutes 50 3 Min. 0 Row 1 Row 2 2017/5/22 .... Pre EVI (S40-12) 740-12 EVI-Stage 2 740-12-V4R4 52 ENCODED VECTOR INDEX (EVI) Given an EVI on table EMPLOYEE keyed on STATE... Set bits in bitmap or return RRN list ...WHERE STATE = ‘ILLINOIS’ Symbol Table Binary search symbol table for key(s) and code(s ) Key Code ARIZONA ARKANSAS CALIFORNIA COLORADO ILLINOIS IOWA KANSAS MISSISSIPPI ... 1 2 3 4 5 6 7 8 ... Index for delivering fast data access in analytical and reporting environments 2017/5/22 Scan vector for code(s) Vector Bitmap 1 17 5 9 2 7 49 49 5 . . . 0 0 1 0 0 0 0 . . . . 0 1 . . . . . . . Row 1 2 3 4 5 6 7 8 9 ... 53 Index ANDing / ORing Example 1 State Workdept EVI EVI Intermediate RRN list 3 5 10 15 1000 AND 1005 (Merge) 1007 3001 3050 State 2017/5/22 SELECT * FROM EMPLOYEE WHERE STATE = ‘MINNESOTA' AND WORKDEPT IN ( 'B01', C01, 'E01') Intermediate RRN list Final RRN list 3 7 10 27 1000 1010 2035 3001 4100 3 10 1000 3001 Workdepts Represents all the local selection Merge/Skip Sequential Plans 54 Index ANDing / ORing Example 2 State Workdept EVI Radix SELECT * FROM EMPLOYEE WHERE STATE = ‘IOWA' OR WORKDEPT IN ( 'B01', C01, 'E01') Intermediate Bitmap Intermediate Bitmap Final Bitmap 0 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 OR 0 0 0 0 (Merge) 0 0 0 1 1 . . . . . . . . . 1 1 1 . . . 0 0 State Workdepts . 2017/5/22 . . . . Represents all the local selection 55 Indexing Strategy • You must create some indexes – Statistics – Implementations • Proactive – Create indexes over primary, foreign key columns and dependent columns – Create indexes for selection and joining – Create indexes for selection, grouping and ordering • Reactive – Create indexes based on optimizer feedback • Visual Explain • Database monitors • Joblog messages • Create indexes based on optimization, implementation, system resources and performance 2017/5/22 56 DB2 for i cardinality The number of elements in a set. • High cardinality = large distinct number of values • Low cardinality = small distinct number of values In general… • A radix index is best when accessing a small set of rows and the key cardinality is high • An encoded vector index is best when accessing a set of rows and the key cardinality is low • Understanding the data and query are key 2017/5/22 57 DB2 for i selectivity The number of records (expressed as a percentage of total records possible) that will be returned returned given a predicate or set of predicates. • High selectivity = large number of records returned • Low selectivity = smaller number of records returned In general… • A radix index is best when accessing a small set of rows, selectivity of predicates is low • An encoded vector index is best when accessing larger set of rows, selectivity of predicates is high 2017/5/22 58 Indexing Strategy – Proactive Basic Approach Radix Indexes • Local selection columns Minimum • Join columns • Local selection columns + join columns • Local selection columns + grouping columns • Local selection columns + ordering columns • Ordering columns + local selection columns Encoded Vector Indexes • Local selection column (single key column of low cardinality) • Join column (data warehouse - star or snowflake schema) EVIs on join columns of my large fact tables, especially foreign key 2017/5/22 59 大纲 • • • • • • DB2 for IBM i 全局概览 DB2 for IBM i 核心特征 DB2 for IBM i 高级功能 DB2 for IBM i 数据库访问 DB2 for IBM i 性能调优 IBM i Navigator基本操作 2017/5/22 60 How Do I Start the IBM i Navigator? • It’s simple! If the system is up, the system database is up • Support for multiple database instances per server (since V5R2) – • Allows you to work on multiple databases on a single system – – • 61 Requires special setup of Independent Auxiliary Storage Pools (IASPs) Expand each database to work at that level No need to "switch" back and forth when working with multiple databases on a specific system See Relational Database Directory entries Explorer-like Interface Folders under each Schema (Library), for each object type. 2017/5/22 62 Managing the Schema – Seeing is believing • • • • • • 63 A list of database objects in the schema is nice, but... Now I understand this App (no wonder we didn't understand it!) Easily see how decisions could affect the schema – I already have five indexes over that table? – Should I remove one? – That view is really over seven tables? – I didn't know we had RI defined over those e.g. application designer creates a test database using the picture and brings it to me for sign off Maps stored on i5/OS Advanced, graphical version of DSPDBR Database Navigator Can easily see relationships for existing schemas, including: Referential Constraints Indexes Views Journals Primary/Unique Keys Aliases Check Constraints Triggers UDRs Table Partitions MQTS 2017/5/22 64 Database Navigator – UDRs User-Defined Relationships can be defined to document links • Not SQL constructs • A way to depict applicationspecific relationships 65 Creating a Schema (called Library pre-V5R3) • CREATE SCHEMA MYSCHEMA; – Organizational construct – Library/Folder • A stored procedure is shipped with i5/OS that can create sample schemas – CALL CREATE_SQL_SAMPLE(‘CORPDATA’); – Contains tables (including sample data), views, aliases, indexes, and RI – CORPDATA is used in the SQL manuals and labs 67 Database Object creation • Create database objects with no storage considerations (including tables!) • Browse function for reusing existing column definitions • Robust Alter Table support – No need for special utilities How do I rebalance indexes? - it’s simple! The database does it for me. New index create options for specifying parallel degree and page size 68 Run SQL Scripts • Create/save scripts for repetitive tasks • Both SQL statements and CL commands • See output for SQL queries • Call stored procedures with result sets and/or OUTput parameters and launch iSeries System Debugger • Do Visual Explain on queries, for performance analysis and tuning • Syntax error highlighting and font chooser • SQL Assistant for building SQL statements • SQL state errors in message box, including second level text • Prompting of CL commands • Ignore drop error messages 69 Run SQL Scripts • • • • • • • • • • • CHK SQL portability New V6R1 Save to SRCPF Save to IFS file Save to UTF8 file Save output to Spread sheet Toggle output to use display heading Undoc result window Launch directly from SQL IFS object Print results Font applies to output Save as for output 70 Show SQL statement from create dialogs • SQL statement is recorded as you fill in dialog • Option to show the SQL statement being built • Great for learning SQL syntax • Allows for statement customization 71 Keeping schema source • DDS - source files - (object source file info maintained) • SQL - Run SQL Statements PC or IFS files • SQL - RUNSQLSTM source files (object source file maintained) 72 Generate the SQL – Regenerate SQL CREATE/ALTER statements directly from the existing objects • Helpful for when the original statements are no longer available • Teach programmers about SQL syntax of objects See what the SQL form of my DDS-created files would be like Include generating short names of objects Do not generate RI and triggers for tables Include SET OPTIONS statement V6R1: Now supported at folder level versus schema or object 2017/5/22 73 Journaling • • 74 Default journal automatically created and started when creating a schema Journal receivers contain journal entries for recovery Table Description Answers questions like: How big is this table/file getting anyway? – Works for indexes and views too! 75 How do I synch up my DB and OS userids? it’s Simple! Group and user authorizations are managed by operating system • • • • • • 76 SQL syntax maps to i5/OS security types Column level security Program vs. object level security Views Exits Auditing Database Loading • Parallel Data load (Import)– Wizard based on CPYFRMIMPF and CPYTOIMPF CL commands – 77 Show Command displays command syntax in Run SQL Scripts Constraint Repair • Works for both check constraints and referential integrity constraints • Show problem rows in table • Able to update column values or delete entire rows • Quick access to check constraint definition for check constraint repair • Quick access to parent table rows for RI constraint repair 78 Reorganize Table Progress Monitoring • New reorganize table options: – Allow suspend (cancel) – Allow online access – Allow changes – Influence when to rebuild access paths • 79 Allow reorganize to be suspended and resumed later Reorganize Table --- Monitoring the Progress • Show percent complete of reorganize and status of indexes as they are reorganized • Attach to an already-running reorganization to see progress • Ability to Suspend an ongoing reorg – even after disconnect of client 80 Manage Index Rebuilds 2017/5/22 Manage index re-creation after Reorgs, Database Restores, etc • Priority Adjusting • Monitor Progress GUI version of EDTRBDAP system command 81 Health Center • Overview to see database object counts by category • View database wide or schema specific – % wildcards can be used • Capture to history file • View history of counts for trending • Environmental Limits 82 New V6R1 Environmental Limits • DB limits within Jobs • Lob Locators • Active Descriptors • Max SQL STMT Size 2017/5/22 83 Health Center • Size and Design Limits views – Size limits are things that change by normal operations – Design limits are things that change by application design • View database wide, schema specific or by object name – % wildcards can be used • Categorized by limit type, includes system limit! • Status indicator for “within percent of limit” – Changeable threshold • • 84 Capture to history file View history of limits for trending Current SQL for a Job • • Launched from Databases folder Find a job for which someone is reporting problems All columns are sortable Shows schema name and program name, service program name, or package name, IP Address, Can view job log, Show SQL Details, or end job Edit SQL launches SQL statement into Run SQL Scripts window, or can go directly to Visual Explain Can See Environment, Statement And Resource Details Refresh allows for monitoring • • • • • • 85 Work Management for Database Jobs • Work with jobs – Find a job for which someone is reporting problems – Customize list by user, job information, and column filtering – Find SQL statement run by the job 86 Who’s Got Those Table Rows Locked? • See what jobs have which rows locked for a given table – Shows the row number locked – Current User column shows “real” userid if from a client – Job information provided – Viewing of row locks also available in Work Management folder for jobs and threads 2017/5/22 87 Live Analysis – Index Advisor Want DB2® to help with your Indexing Strategy? New system-wide repository of DB2 Advised Indexes – System view makes it easier to create index with biggest impact – Gauge benefits of index creation based on frequency, last occurrence, common keys – Available at the database, schema, or individual table level Enhanced Index Advice (SQE-Only) Ability to view SQL statements that caused index advice (SQE-Only) Index Advice Condenser to ease DB2 Performance tuning – http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0701forstie/ 2017/5/22 88 Live Analysis – SQL Plan Cache Easy analysis of current SQL workloads … Without overhead of database monitor Advanced filtering to quickly narrow focus to the most interesting SQL – By number of executions – By DB2 object – … Easy access to Visual Explain for detailed analysis Archive capability into snapshot for future comparisons SQE-Only 2017/5/22 89 SQL Performance Monitor Toolset Have an existing application and want to capture the SQL activity and analyze it? • Start / Stop database monitor – Pre & Post-Filtering – Summary or Detailed – Import data from other systems • Areas to watch: – – – – • • Index advised Full opens vs. pseudo opens Access plan rebuilds Index creates Recommended during application tuning & problem determination Keep a baseline monitor run for comparison 2017/5/22 90 SQL Performance Monitor Filtering Minimize system-impact by reducing amount of data collected – – – – Collect data for a specific user Collect data for a collection of jobs (QZDASO*) Collect data for longer-running SQL requests … Enables quicker analysis & identification of SQL performance bottlenecks Post-Filtering available for Visual Explain interface 2017/5/22 91 SQL Performance Monitor Analysis • Summary view to more simply decide, if the monitor has attributes that require deeper analysis • Integrated analysis with ability to launch other performance tools from current analysis view Dashboard Summary Drill-Thru Analysis 2017/5/22 92 SQL Performance Monitor Comparison • Want to know if an application change, system change or PTF altered your DB2 performance? – Compare current SQL performance “profile” with archived monitor data – See if the optimizer’s implementation of an SQL statement has changed – Can be long-running operation for large database monitor collections – Can do High level compare of two or more monitors 93 Analyze SQL Performance Monitors with Visual Explain • Select existing detailed SQL Performance Monitor • Show list of SQL statements captured by the monitor • Can import monitor data from other systems • Can narrow down analysis with postfilters 94 Visual Explain • Active window - Look at DB objects (indexes, constraints, etc.) involved • Attributes show all the settings that impact query optimizer – – • Icon Highlighting to speed analysis – – • 95 Labels include optimizer’s estimates SQE plans show decomposed SQL statement for each operation Expensive nodes Index Advised nodes Debug messages help bridge to picture VE Explain While Running • Allows you to see actual runtime feedback in VE diagram THIS IS VERY COOL!! 96 New V6R1 Index Advisor • Advisor shows indexes that would help specific query (if any) • Use advice along with Index Analyzer for the table to determine if index would be helpful • Create button prompts Create Index dialog with recommended columns pre-filled 2017/5/22 97 Statistics • Advisor in Visual Explain shows statistics that would help specific query • How do I run statistics? – • 98 It’s simple! Statistics automatically created/maintained by database Cardinality, Most common values Evaluating your Indexes & MQTs Is an Index or MQT providing value? • • Launch Evaluators by right-clicking on table and select Show Index or MQT Scroll Right to find the interesting query optimizer usage statistics – – • Stats updated for all query interfaces – MQTs are SQL-only Includes all objects the optimizer evaluates whether created by DDS or SQL Index Evaluator available on V5R2 & V5R3 via PTF – statistics not updated until the PTFs are applied 2017/5/22 99 Enhanced Index management From Index Folder I can now: • See Owning Index Name • See/ Reset Query Usage Stats • Show Statements from plan cache that use this index 2017/5/22 New V6R1 100 Analyze Performance of SQL Programs & Packages • Explain SQL is equivalent to PRTSQLINF CL command • Can Explain SQL for functions, procedures, triggers, and packages • Can Explain SQL for programs by going down the File System folder • How do I rebind my application packages? – It’s simple! The database does it for you automatically. 10 1 SQL and External Triggers • Easily create External and SQL Triggers with new wizard • Also added Instead of triggers over views 2017/5/22 102 Stored Procedures • Easily create External (including Java) and SQL stored procedures • A List of example SQL statements makes creating SQL procedures easier 2017/5/22 103 SQL Graphical Debug • Starts with SQL *SOURCE Debug View – Specifying DBGVIEW=*SOURCE on the SET OPTION clause in your SQL procedure, function, or trigger – Specify a *SOURCE value for DBGVIEW parameter on the RUNSQLSTM (Run SQL Statements) CL command • Graphical iSeries System Debugger – Part of the Toolbox for Java – Typical debugger capabilities – Can debug SQL procedures, functions, and triggers that run on System i5 family 2017/5/22 104 iSeries System Debugger - example 2017/5/22 105 Ease of Database Management Typical DBA Tasks DB2 for i5/OS Manage DASD Space Allocation Automated Review Table Space Allocations & Extents Automated Review & Balance Indexes Automated Application Rebinding Automated Maintain Database Integrity Automated Update Database Statistics Automated Synchronized OS & DB User Security Automated Reload Data for Hardware & Software Upgrades Automated Load Data into Database System i Navigator Build and Manage DB Backup & Recovery System i Navigator Create & Review Indexes for Tables System i Navigator / Automated Index Advisor Performance Analysis & Tuning (DB & System) System i Navigator Create and Maintain DB Schema System i Navigator & 3rd Party Tools Automated DB Performance Profiling 3rd Party Tools Advanced DB Performance Analysis & Tuning System i Navigator 3rd Party Tools Data Replication & Consolidation Multiple IBM & 3rd Party Products Warehouse common schema • • • • • • • • • • • • • • • • TB_WAREHOUSE TB_TRANSACTION_IDENTIFICATION TB_STOCK (Partitioned Table) TB_STATE_IDENTIFICATION TB_ORDERLINE TB_ORDER (Partitioned Table) TB_NEWORDER TB_ITEM_CATEGORY_IDENTIFICATION TB_ITEM TB_INCOME_RANGE TB_HISTORY (Partitioned Table) TB_ETHNIC_GROUP TB_DISTRICT TB_CUSTOMER_ACCOUNT TB_CUSTOMER (Partitioned Table) TB_AGE_GROUP 2017/5/22 107 Questions & Answers? 2017/5/22 108