* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download DataServer Best Practices and Performance Considerations with
Survey
Document related concepts
Serializability wikipedia , lookup
Microsoft Access wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Concurrency control wikipedia , lookup
Oracle Database wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Clusterpoint wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Database model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Transcript
DataServer Best Practices and Performance Considerations with OpenEdge 11 28th Jun 2016 Sachin Garg – OpenEdge NOTES TO PRESENTER: There is a great deal of content per slide making them difficult to read. Particularly, on the slides with a dark grey background and text boxes in neon green. Consider using a lighter green as noted, increasing pt size of text box content from 12 to 16, and spacing the content across more slides to make it more easily visible. Thank you! 2 Best practices generally revolve around techniques used to optimize the development effort, deployment effort, maintenance effort and foremost, the performance improvements that one can get out of his/her application Agenda Introduction Best Practices Performance Considerations Upcoming Features 4 Introduction: Architectural View of DataServer Components ABL Application Run time interpreter ABL/DataServer Client ~~~~~ compiler ~~~~~ Server Probrkr/ Ubroker R-code Driver (OCI/ODBC) Client-n Dataserver Server-n Schema Holder 5 Data Source DataServer Components Client Component ABL, AppServer, WebSpeed DataServer (self-serviced) Server component DataServer for MSS/Oracle DataServer for ODBC (retired) Data Access components ODBC/OCI driver DD Wire Protocol for MSSQL Server/DB2/Sybase SQL Server Native Client Driver v9/10/11 Oracle client 10g/11g/12c Database components Oracle 10g/11g/12c MS SQL Server 2005/2008R2/2012/2014 DB2/UDB and DB2/AS400, Sybase 6 Schema holder deployment models Schema holder is central to your DataServer application deployment model. Two methods of deployment Schema Migration Schema Pull Schema Migration: Use Schema Migration model if OpenEdge schema needs to be distributed to multiple copies of foreign data source. Any incremental changes will be done at single source copy and can be distributed to multiple copies of schema holder and foreign database Schema Pull Use Schema Pull to create schema holder definitions of foreign database objects. There is no single source copy of OpenEdge database. Schema Pull is a server based objects OE application model e.g. stored procedures, views are foreign database specifics. New independent schema pull in batch mode utility makes the schema pull process simple. 7 Agenda Introduction Best Practices Performance Considerations Upcoming Features 8 ROWID Selection (MSS DataServer) Use Best ROWID choice for your application There are several methods to choose ROWID, weigh over each choice and choose one that best suit to your application Some of the guidelines are: Try OpenEdge Primary Index for ROWID selection Provides default sort order if other sort criteria is unspecified Similarly Clustered Index provides default sort order in MSS If OE primary is unique and mandatory, can become MSS Primary MSS Primary are made implicit clustered if no explicit clustered index Clustered index provides benefit of native SQL Server capability Select ‘Best’ ROWID Let Progress choose the best index as ROWID Scan through all the available indices Unique single integer component mandatory column Go for default (If OE primary is not unique and cannot serve as ROWID) PROGRESS_RECID provides the default choice As ROWID Computed column can provide better choice over trigger based PROGRESS_RECID and can be easily mapped to MSS primary Code reusability against schema changes - R-code safeguarded For foreign position change Foreign column position can be altered No r-code dependency on physical position R-code compiled before the schema change will continue to run Makes development independent of deployment Available from OE 11.0 onwards Updated Schema Original Schema Name Name forpos logpos Custnum 1 1 Name 2 2 3 City 4 3 4 State 3 4 Custnum 1 1 Name 2 2 City 3 State 4 R-code sh2 Foreign DB For adding a new field Customer Custnum Name 1 2 City 3 sh1 Customer Custnum Name 1 2 Fail to run R-code Can add new field to existing schema No CRC mismatch error R-code compiled before the change will continue to run New field can be added using delta.SQL or schema PULL Available from OE 11.4 onwards forpos logpos Customer City Custnum Name 1 2 3 sh2 Customer run City Custnum Name 3 1 2 Programming Considerations: Record scope and Locks Record Availability Procedure getcust.p DEFINE BUFFER bcust for Customer. CREATE Customer. ASSIGN custnum = 100 FIND bcust WHERE bcust.custnum = 100 DISP bcust. Procedure getcust.p DEFINE BUFFER bcust for Customer. CREATE Customer. ASSIGN custnum = 100 VALIDATE Customer. /* or RELEASE or GET RECID/ROWID. */ FIND bcust WHERE bcust.custnum = 100 DISP bcust. 11 Transaction Boundaries In DataServer locks are released at the transaction boundary DEFINE BUFFER b1_state FOR state. DEFINE BUFFER b2_state FOR state. /* Transaction 1 (session-1) */ DO TRANSACTION: FIND b1_state WHERE b1_state.State = 'CA' EXCLUSIVE-LOCK. DISPLAY b1_state.StateName with frame f1. END. /* Transaction 2 (session-2) */ PAUSE MESSAGE "pause until press any key". /* Transaction 3 (session-1) */ DO TRANSACTION: FIND b2_state WHERE b2_state.State = 'CA' EXCLUSIVE-LOCK. DISPLAY b2_state.StateName with frame f2. END. 12 Secure your identity and data: Network security Username/Password Encryption (OE 11.4 onwards) Enabled for Dataserver Broker/Network mode DataServer client sends Username and Password in the encrypted form DataServer server decrypts the credentials Available from 11.4 onwards Use SSL enabled DataServer connection (OE 11.6 onwards) Provide SSL enabled network connection from DataServer client to DataServer Server SSL protocol provide secure access of your data on both the side of network. Strong authentication and encryption of data in-transit Usage: Start client or agent with –ssl parameter OpenSSL library is upgraded to latest version to avoid any security threats like POODLE etc. 13 Programming Considerations Fixed Vs. Variable Length character Progress character is variable length SQL needs a length for character column Schema Migration uses format x(<n>) for SQL width However x(8) is just the display format Run dbtool to adjust the SQL Width prior to schema migration BLANKDEFAULT Maps OE default initial value to Oracle blank space To distinguish empty string Oracle NULLs as Oracle see them as same 14 Handling lock conditions in DataServers Problem: I am trying to create/update a record in a table or trying to read a record with a lock, but getting lock timed out Solution: Provide small window to wait on server for lock acquisition to avoid frequent roundtrip Use –Dsrv PRGRS_NATIVE_LOCKWAIT, <timeout> for server timeout period. Where <timeout> is in milliseconds. Default is NO-WAIT -lkwtmo is client lock timeout period. Default is 180 seconds Support added in 10.2B08 and 11.x in Oracle DataServer. MSS DataServer has support since 10.1B -Dsrv PRGRS_NATIVE_LOCKWAIT,<timeout> translation: Oracle DS: WAIT <seconds> MSS DS: (min precision is 1 second) SET LOCK_TIMEOUT <milliseconds> 15 Misc. best practices Schema Deployment: Deploy schema holder local to the client machine Use –cache option to cache the schema definition local to the machine. R-code reusability Use views over tables if schema changes are unavoidable Use alias for logical database for r-code reusability ROWID selection: Try ROWID mapped to primary and clustered index. Create your primary index first Else OpenEdge assumes the first index you create is primary index. Logging: Use enhanced logging parameters to manage the huge log files -logthreshold will create a new dataserv.lg log file once the existing log file hit the threshold value. UNIQUE_DSLOG will generate separate dataserv.lg log files for each agent or client process. 16 Agenda Introduction Best Practices Performance Considerations Upcoming Features 17 Performance Considerations Avoid FINDs Replace FIND FIRST with FOR FIRST if a single record desired Cursor remains in open state for the entire session Server prepares a cache of record identifiers and then retrieve individual rows Ignores the field list. always gets the entire record Skip the schema verification • Schema holder is supposed to be in sync with foreign database schema • Configured though connection parameter -Dsrv skip-schema-check • Unmatched data definitions might cause data corruptions • Make sure to perform an Add/Update to sync schema holder with Database Dataserv.lg :----------Skipping schema check for table CUSTOMER---------- Performance Considerations Field Lists (ABL Tuning) Make sure to include all fields you reference. This compiles, but produce run-time error. Reduces the network traffic Only works with NO-LOCK query. EXCLUSIVE-LOCK disables the FIELD LIST FOR EACH Customer FIELDS(custnum name) NO-LOCK: FIND FIRST Order WHERE Order.st = Customer.st. END. QUERY-TUNING (NO-FIREHOSE-CURSOR) Long running query Only a few records are desired out of very large result set Uses server side/Block cursor. Firehose cursor/Default result set is disabled. DEFINE VARIABLE iter as INTEGER. FOR EACH Customer QUERY-TUNING (NOFIREHOSE-CURSOR) : iter = iter + 1. IF iter = 20 THEN LEAVE. FIND FIRST Order OF Customer. END. Performance Considerations Shadow Column Provides case insensitive data access from a case sensitive database choose shadow column migration option to generate a surrogate column Hidden, Indexed, Stores a copy of base value in uppercase Name are prefixed by _S# in MSS and by U## in Oracle DS Bulk Insert Inserts records in batches Several hundred records are inserted into ORACLE as a single array insert operation Available Both as Data Administrator tool and ABL constructs Limitations: LOBs are inserted as empty. Triggers may not fire as desired. Syntax: RUN STORED-PROC SEND-SQL-STATEMENT h1= proc-handle (“—Bulk-insert Start <n>”) RUN STORED-PROC SEND-SQL-STATEMENT h2 = proc-handle (“—Bulk-insert End”) <n> denotes number of records per transaction Performance Considerations Following JOINs now qualifies for server side Joins INNER JOIN (Since OE 9.x) FOR EACH Customer, EACH Order OF Customer OUTER-JOIN (Since 11.0.0, Refined in 11.2.0) FOR EACH Customer, EACH Order OF Customer OUTER-JOIN Dynamic query JOIN (Since 11.0.0, Refined in 11.2.0 and later releases) FIRST/LAST join (Since 11.3.0, Refined in 11.4.0) FOR EACH Customer, FIRST Order OF Customer Performance Considerations FIRST/LAST server-side Join (know your data) Implemented as correlated subquery in OE 11.4. FOR EACH Customer, FIRST Order OF Customer: Database executes subquery for every evaluation of outer query DISP Customer.CustNum Order.OrderNum. END. Query performance is good if records are evenly distributed in child table based on the parent table join key join by client is efficient if child table has tens of thousands matching record for a few parent records Limitations: Following conditions will turn the query as join by client – Date Field in WHERE clause – Program variable used in WHERE clause – BY clause SQL Translation: SELECT <Customer Table Field List> <Order Table Field List> FROM Customer T0, Order T1 WHERE Customer.CustNum = Order.CustNum AND Order.OrderNum = (SELECT MIN (OrderNum) FROM Order WHERE Order.OrderNum = Customer.CustNum) Performance Considerations Server side OUTER-JOIN An OUTER-JOIN query is offloaded to foreign database engine by default starting OE 11.0.0 Improves the performance as smaller result set is sent to client in a single database request E.g. FOR EACH Customer, (Customer 10 records) EACH Order OF Customer OUTER-JOIN. (Order table 200 records ) Join By Client: It works like a nested loop. i.e. for each customer record database request, there is a database request for all matching order record in a loop Join By Server: Single database request. Cost based optimizer capability will choose the best data access plan OUTER-JOIN request is served by USE_MERGE hint USE_MERGE hint sorts the records in the join key order, which is a perquisite for ABL client. Is HASH_JOIN better? Produce unsorted results Can overridden by using QUERY-TUNING (HINT ‘USE_HASH’) Performance Considerations WHERE clause If not necessary, avoid using ABL specific function in WHERE clause They get resolved by the client Replace with constructs understood by server e.g. Instead of LOOKUP (cust-num, “1,2,3”), use cust-num = 1 OR cust-num = 2 OR cust-num = 3 • don’t get too fancy with ABL functions WHERE cust-num = integer (“123”) is better than WHERE STRING (cust-num) = “123” Try to use covering index for server side sorting of the results. e.g. WHERE col1 = <val> and col2 = <val> BY col1, BY col2 will sort the results on server Performance Considerations Stored Procedures and SEND-SQL-STATEMENT (DBMS) • Dataserver supports SQL procedures, direct/pass-thru SQL statement from ABL • Reduced network traffic, server side processing of SQL statements • Downside : code is database dependent Syntax: RUN STORED-PROC <procedure> [<int> = PROC-HANDLE] NO-ERROR CLOSE STORED-PROC <procedure> WHERE PROC-HANDLE = <int>. Index Reposition (ABL Tuning) Speedup the random access within a query Can jump to any record position in any direction Records are first looked up in the client side cache A new result set is built from the new rowid position Syntax : DEFINE QUERY q1 FOR order SCROLLING. OPEN QUERY q1 FOR EACH order INDEXEDREPOSITION. REPOSITION q1 to ROWID <rowid>. GET NEXT q1. Agenda Introduction Best Practices Performance Considerations Upcoming Features 26 A glance to OpenEdge DataServer 11.6 Features PASOE support for DataServers Multithreaded self-service DataServer Multi session appserver agent DataDirect Wire Protocol Driver for MSSQL Server Benchmark shows better performance Legacy Wire Protocol driver support will be discontinued SSL support for DataServers DataServer network is SSL enabled Both for DataServer broker (_probrkr) and Unified Broker RECID area restructuring Variable size New ROWID format independent of ZPRGRS_RECID_BUF_SIZE Upcoming Features/Enhancements Focus is on Joins Improvements Server side Index reposition for JOINs in Oracle DataServer GET NEXT exclusive-lock for JOIN query in MSS DataServer FIRST/LAST join in MSS DataServer Result Set Binding for Join records Nonetheless other features are also crucial • Native sequence support for ROWID in MSS DataServer • Word Indexing Support in MSS DataServer • Save points support in MSS DataServer • Field List Support for Exclusive-Lock query