Download DataServer Best Practices and Performance Considerations with

DataServer Best Practices and
Performance Considerations with
OpenEdge 11
28th Jun 2016
Sachin Garg – OpenEdge
NOTES TO PRESENTER:
 There is a great deal of content per slide making them difficult to read. Particularly, on the
slides with a dark grey background and text boxes in neon green. Consider using a lighter
green as noted, increasing pt size of text box content from 12 to 16, and spacing the content
across more slides to make it more easily visible. Thank you!
2
Best practices generally revolve around
techniques used to optimize the development
effort, deployment effort, maintenance effort
and foremost, the performance improvements
that one can get out of his/her application
Agenda
 Introduction
 Best Practices
 Performance Considerations
 Upcoming Features
4
Introduction: Architectural View of DataServer Components
ABL
Application
Run time
interpreter
ABL/DataServer
Client
~~~~~
compiler
~~~~~
Server
Probrkr/
Ubroker
R-code
Driver
(OCI/ODBC)
Client-n
Dataserver
Server-n
Schema
Holder
5
Data
Source
DataServer Components




Client Component
 ABL, AppServer, WebSpeed
 DataServer (self-serviced)
Server component
 DataServer for MSS/Oracle
 DataServer for ODBC (retired)
Data Access components
 ODBC/OCI driver
 DD Wire Protocol for MSSQL Server/DB2/Sybase
 SQL Server Native Client Driver v9/10/11
 Oracle client 10g/11g/12c
Database components
 Oracle 10g/11g/12c
 MS SQL Server 2005/2008R2/2012/2014
 DB2/UDB and DB2/AS400, Sybase
6
Schema holder deployment models
 Schema holder is central to your DataServer application deployment model.
 Two methods of deployment
 Schema Migration
 Schema Pull
 Schema Migration:
 Use Schema Migration model if OpenEdge schema needs to be distributed to multiple copies of foreign
data source.
 Any incremental changes will be done at single source copy and can be distributed to multiple copies of
schema holder and foreign database
 Schema Pull
 Use Schema Pull to create schema holder definitions of foreign database objects. There is no single
source copy of OpenEdge database.
 Schema Pull is a server based objects OE application model e.g. stored procedures, views are foreign
database specifics.
 New independent schema pull in batch mode utility makes the schema pull process simple.
7
Agenda
 Introduction
 Best Practices
 Performance Considerations
 Upcoming Features
8
ROWID Selection (MSS DataServer)
Use Best ROWID choice for your application
There are several methods to choose ROWID, weigh over each
choice and choose one that best suit to your application
Some of the guidelines are:
 Try OpenEdge Primary Index for ROWID selection





Provides default sort order if other sort criteria is unspecified
Similarly Clustered Index provides default sort order in MSS
If OE primary is unique and mandatory, can become MSS Primary
MSS Primary are made implicit clustered if no explicit clustered index
Clustered index provides benefit of native SQL Server capability
 Select ‘Best’ ROWID



Let Progress choose the best index as ROWID
Scan through all the available indices
Unique single integer component mandatory column
 Go for default (If OE primary is not unique and cannot serve as ROWID)


PROGRESS_RECID provides the default choice As ROWID
Computed column can provide better choice over trigger based
PROGRESS_RECID and can be easily mapped to MSS primary
Code reusability against schema changes - R-code
safeguarded
 For foreign position change
 Foreign column position can be altered
 No r-code dependency on physical position
 R-code compiled before the schema change will
continue to run
 Makes development independent of deployment
 Available from OE 11.0 onwards
Updated Schema
Original Schema
Name
Name
forpos logpos
Custnum
1
1
Name
2
2
3
City
4
3
4
State
3
4
Custnum
1
1
Name
2
2
City
3
State
4
R-code
sh2
Foreign DB
 For adding a new field
Customer
Custnum
Name
1
2
City
3
sh1
Customer
Custnum
Name
1
2
Fail to run
R-code
 Can add new field to existing schema
 No CRC mismatch error
 R-code compiled before the change will
continue to run
 New field can be added using delta.SQL or
schema PULL
 Available from OE 11.4 onwards
forpos logpos
Customer
City
Custnum
Name
1
2
3
sh2
Customer
run
City
Custnum
Name
3
1
2
Programming Considerations: Record scope and Locks
 Record Availability
Procedure getcust.p
DEFINE BUFFER bcust for Customer.
CREATE Customer.
ASSIGN custnum = 100
FIND bcust WHERE bcust.custnum = 100
DISP bcust.
Procedure getcust.p
DEFINE BUFFER bcust for Customer.
CREATE Customer.
ASSIGN custnum = 100
VALIDATE Customer. /* or RELEASE or GET RECID/ROWID. */
FIND bcust WHERE bcust.custnum = 100
DISP bcust.
11
Transaction Boundaries

In DataServer locks are released at the transaction boundary
DEFINE BUFFER b1_state FOR state.
DEFINE BUFFER b2_state FOR state.
/* Transaction 1 (session-1) */
DO TRANSACTION:
FIND b1_state WHERE b1_state.State = 'CA' EXCLUSIVE-LOCK.
DISPLAY b1_state.StateName with frame f1.
END.
/* Transaction 2 (session-2) */
PAUSE MESSAGE "pause until press any key".
/* Transaction 3 (session-1) */
DO TRANSACTION:
FIND b2_state WHERE b2_state.State = 'CA' EXCLUSIVE-LOCK.
DISPLAY b2_state.StateName with frame f2.
END.
12
Secure your identity and data: Network security

Username/Password Encryption (OE 11.4 onwards)
 Enabled for Dataserver Broker/Network mode
 DataServer client sends Username and Password in the encrypted form
 DataServer server decrypts the credentials
 Available from 11.4 onwards

Use SSL enabled DataServer connection (OE 11.6 onwards)
 Provide SSL enabled network connection from DataServer client to DataServer Server
 SSL protocol provide secure access of your data on both the side of network.
 Strong authentication and encryption of data in-transit
 Usage: Start client or agent with –ssl parameter
 OpenSSL library is upgraded to latest version to avoid any security threats like POODLE etc.
13
Programming Considerations

Fixed Vs. Variable Length character
 Progress character is variable length
 SQL needs a length for character column
 Schema Migration uses format x(<n>) for SQL width
 However x(8) is just the display format
 Run dbtool to adjust the SQL Width prior to schema migration

BLANKDEFAULT
 Maps OE default initial value to Oracle blank space
 To distinguish empty string Oracle NULLs as Oracle see them as same
14
Handling lock conditions in DataServers
 Problem:
I am trying to create/update a record in a table or trying to read a record with a lock,
but getting lock timed out
 Solution:
 Provide small window to wait on server for lock acquisition to avoid frequent roundtrip
 Use –Dsrv PRGRS_NATIVE_LOCKWAIT, <timeout> for server timeout period.

Where <timeout> is in milliseconds. Default is NO-WAIT
 -lkwtmo is client lock timeout period. Default is 180 seconds
 Support added in 10.2B08 and 11.x in Oracle DataServer. MSS DataServer has support since 10.1B
-Dsrv PRGRS_NATIVE_LOCKWAIT,<timeout> translation:
Oracle DS: WAIT <seconds>
MSS DS:
(min precision is 1 second)
SET LOCK_TIMEOUT <milliseconds>
15
Misc. best practices
 Schema Deployment:
 Deploy schema holder local to the client machine
 Use –cache option to cache the schema definition local to the machine.
 R-code reusability
 Use views over tables if schema changes are unavoidable
 Use alias for logical database for r-code reusability
 ROWID selection:
 Try ROWID mapped to primary and clustered index.
 Create your primary index first
 Else OpenEdge assumes the first index you create is primary index.
 Logging:
 Use enhanced logging parameters to manage the huge log files
-logthreshold will create a new dataserv.lg log file once the existing log file hit the threshold value.
UNIQUE_DSLOG will generate separate dataserv.lg log files for each agent or client process.
16
Agenda
 Introduction
 Best Practices
 Performance Considerations
 Upcoming Features
17
Performance Considerations
 Avoid FINDs




Replace FIND FIRST with FOR FIRST if a single record desired
Cursor remains in open state for the entire session
Server prepares a cache of record identifiers and then retrieve individual rows
Ignores the field list. always gets the entire record
 Skip the schema verification
• Schema holder is supposed to be in sync with foreign database schema
• Configured though connection parameter -Dsrv skip-schema-check
• Unmatched data definitions might cause data corruptions
• Make sure to perform an Add/Update to sync schema holder with Database
Dataserv.lg :----------Skipping schema check for table CUSTOMER----------
Performance Considerations
 Field Lists (ABL Tuning)





Make sure to include all fields you reference.
This compiles, but produce run-time error.
Reduces the network traffic
Only works with NO-LOCK query.
EXCLUSIVE-LOCK disables the FIELD LIST
FOR EACH Customer FIELDS(custnum name) NO-LOCK:
FIND FIRST Order WHERE Order.st = Customer.st.
END.
 QUERY-TUNING (NO-FIREHOSE-CURSOR)




Long running query
Only a few records are desired out of very large result set
Uses server side/Block cursor.
Firehose cursor/Default result set is disabled.
DEFINE VARIABLE iter as INTEGER.
FOR EACH Customer QUERY-TUNING (NOFIREHOSE-CURSOR) :
iter = iter + 1.
IF iter = 20 THEN LEAVE.
FIND FIRST Order OF Customer.
END.
Performance Considerations
 Shadow Column




Provides case insensitive data access from a case sensitive database
choose shadow column migration option to generate a surrogate column
Hidden, Indexed, Stores a copy of base value in uppercase
Name are prefixed by _S# in MSS and by U## in Oracle DS
 Bulk Insert



Inserts records in batches
Several hundred records are inserted into ORACLE as a single array insert operation
Available Both as Data Administrator tool and ABL constructs
Limitations:


LOBs are inserted as empty.
Triggers may not fire as desired.
Syntax:
RUN STORED-PROC SEND-SQL-STATEMENT h1= proc-handle (“—Bulk-insert Start <n>”)
RUN STORED-PROC SEND-SQL-STATEMENT h2 = proc-handle (“—Bulk-insert End”)
<n> denotes number of records per transaction
Performance Considerations
 Following JOINs now qualifies for server side Joins
 INNER JOIN (Since OE 9.x)
FOR EACH Customer,
EACH Order OF Customer
 OUTER-JOIN (Since 11.0.0, Refined in 11.2.0)
FOR EACH Customer,
EACH Order OF Customer OUTER-JOIN
 Dynamic query JOIN (Since 11.0.0, Refined in 11.2.0 and later releases)
 FIRST/LAST join (Since 11.3.0, Refined in 11.4.0)
FOR EACH Customer,
FIRST Order OF Customer
Performance Considerations
 FIRST/LAST server-side Join (know your data)
 Implemented as correlated subquery in OE 11.4.
FOR EACH Customer,
FIRST Order OF Customer:
 Database executes subquery for every evaluation of
outer query
DISP Customer.CustNum Order.OrderNum.
END.
 Query performance is good if records are evenly distributed
in child table based on the parent table join key
 join by client is efficient if child table has tens of thousands matching
record for a few parent records
 Limitations: Following conditions will turn the query as
join by client
– Date Field in WHERE clause
– Program variable used in WHERE clause
– BY clause
SQL Translation:
SELECT <Customer Table Field List> <Order Table
Field List> FROM Customer T0, Order T1 WHERE
Customer.CustNum = Order.CustNum AND
Order.OrderNum = (SELECT MIN (OrderNum) FROM
Order WHERE Order.OrderNum = Customer.CustNum)
Performance Considerations
 Server side OUTER-JOIN
 An OUTER-JOIN query is offloaded to foreign database engine by default starting OE 11.0.0
 Improves the performance as smaller result set is sent to client in a single database request
 E.g. FOR EACH Customer, (Customer 10 records)
EACH Order OF Customer OUTER-JOIN. (Order table 200 records )
Join By Client: It works like a nested loop. i.e.
for each customer record database request, there is a database request for all matching order record in a loop
Join By Server: Single database request.
 Cost based optimizer capability will choose the best data access plan
 OUTER-JOIN request is served by USE_MERGE hint
 USE_MERGE hint sorts the records in the join key order, which is a perquisite for ABL client.
 Is HASH_JOIN better? Produce unsorted results
 Can overridden by using QUERY-TUNING (HINT ‘USE_HASH’)
Performance Considerations
 WHERE clause



If not necessary, avoid using ABL specific function in WHERE clause
They get resolved by the client
Replace with constructs understood by server e.g.
Instead of LOOKUP (cust-num, “1,2,3”),
use cust-num = 1 OR cust-num = 2 OR cust-num = 3
•
don’t get too fancy with ABL functions

WHERE cust-num = integer (“123”) is better than WHERE STRING (cust-num) =
“123”
Try to use covering index for server side sorting of the results. e.g.

WHERE col1 = <val> and col2 = <val> BY col1, BY col2 will sort the results on server
Performance Considerations
 Stored Procedures and SEND-SQL-STATEMENT (DBMS)
• Dataserver supports SQL procedures, direct/pass-thru SQL statement from ABL
• Reduced network traffic, server side processing of SQL statements
• Downside : code is database dependent
Syntax:
RUN STORED-PROC <procedure> [<int> = PROC-HANDLE] NO-ERROR
CLOSE STORED-PROC <procedure> WHERE PROC-HANDLE = <int>.
 Index Reposition (ABL Tuning)




Speedup the random access within a query
Can jump to any record position in any direction
Records are first looked up in the client side cache
A new result set is built from the new rowid position
Syntax :
DEFINE QUERY q1 FOR order SCROLLING.
OPEN QUERY q1 FOR EACH order INDEXEDREPOSITION.
REPOSITION q1 to ROWID <rowid>.
GET NEXT q1.
Agenda
 Introduction
 Best Practices
 Performance Considerations
 Upcoming Features
26
A glance to OpenEdge DataServer 11.6 Features
 PASOE support for DataServers
 Multithreaded self-service DataServer
 Multi session appserver agent
 DataDirect Wire Protocol Driver for MSSQL Server
 Benchmark shows better performance
 Legacy Wire Protocol driver support will be discontinued
 SSL support for DataServers
 DataServer network is SSL enabled
 Both for DataServer broker (_probrkr) and Unified Broker
 RECID area restructuring
 Variable size New ROWID format
 independent of ZPRGRS_RECID_BUF_SIZE
Upcoming Features/Enhancements
 Focus is on Joins Improvements
 Server side Index reposition for JOINs in Oracle DataServer
 GET NEXT exclusive-lock for JOIN query in MSS DataServer
 FIRST/LAST join in MSS DataServer
 Result Set Binding for Join records
 Nonetheless other features are also crucial
• Native sequence support for ROWID in MSS DataServer
• Word Indexing Support in MSS DataServer
• Save points support in MSS DataServer
• Field List Support for Exclusive-Lock query