Download Siebel Systems & Microsoft

SQL Server: Performance
& Balanced System Design
By Frank McBath
[email protected]
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Dealing with Complexity
 Key Theme –
“Simple but NOT simplistic”
 Static configuration parameters replaced with dynamic algorithms
employing adaptive feedback
 Administrative control retained to manage system wide resources
 You can still constrain the amount of memory used by SQL Server – if you
want to
Dynamic Control
Dynamic control algorithms
maintain “near optimal”
values in response to workload
Control
Value
If you tune at this point
in time you pick this
value.
Maybe you pick
2 values and reconfigure
system for each “use”
Time
Instantaneous optimal
value of control value
Dynamic Control Analogy

Ignition Timing:


Vacuum Advance:



user controlled by lever Classic “knob”
Uses simple feedback
Major advance
Full Feedback:



Continuously adjusted
Monitors: temp, speed,
engine response, etc.
Memory, CPU & Clustering Limits
Windows Server 2003 and SQL Server 2000
Windows Server
2003,
Standard Edition
Windows Server 2003,
Enterprise Edition
Windows Server 2003,
Datacenter Edition
32-bit
32-bit
32-bit
RAM (GB)
2
2
2
CPU
4
4
4
Max
Resource
Supported
SQL Server 2000,
Standard
Edition
64-bit
64-bit
1 With Address
Windowing
Extensions
(AWE) in SQL
Server 2000
Cluster
Nodes
SQL Server 2000,
Enterprise Edition
3
32
CPU
4
8
32
4
4
Cluster
Nodes
SQL Server 2000,
64-bit Edition
1
RAM (GB)
RAM (GB)
64
1
64
512
CPU
8
64
Cluster
Nodes
8
8
Scalability

Optimized for Windows Server 2003 and Itanium

Great performance



Manageability

T-SQL code-compatibility with SQL Server 2000

8 node clustering support

Same on-disk format as 32-bit for easy migration
One setup for database & OLAP based on Windows
Installer technology
Compelling alternative to expensive Unix solutions

Cost Savings
Large memory addressability (up to 32 TB)
Nearly unlimited virtual memory (up to 8 TB)
I/O savings due to larger memory buffer pools

The highly scalable database platform for memory
intensive, performance-critical business applications
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
PSPP Results as of Oct 26, 2003

10/08/03 - 5,000 Concurrent User on Unisys ES7000 server and Microsoft SQL Server 2000

10/07/03 - 32,000 Concurrent User on HP-UX servers

10/07/03 -12,000 Concurrent User on HP ProLiant and Integrity servers and Microsoft SQL
Server 2000 (64-bit)

06/03/03 - 10,000 Concurrent User on HP-UX servers

04/24/03 - 30,000 Concurrent User on Unisys ES7000 / 2000 Series of servers and Microsoft
SQL Server 2000 (64-bit)

02/05/03 - 5,000 Concurrent User on IBM eServer xSeries and Microsoft SQL Server 2000

10/21/02 - 4,500 Concurrent User on IBM eServer xSeries and IBM DB2 UDB

06/24/02 - 30,000 Concurrent User on IBM eServer pSeries and IBM DB2 UDB
World-class performance
30,000 concurrent Users
A typical PSPP environment: HP 12K
12,000 User Benchmark on HP/Windows/SQL64 – resource utilization
Niode
Functional Use
Average CPU (%)
Utilization
Average Mem ory
Utilization (MB)
4 x Proliant DL760
Web Server – Application Requests
8%
600
3 x Proliant BL20p
Web Server – Application Requests
7%
500
1 x Proliant DL760
Web Server – HTTP Adapter, WF
9%
400
1 x Proliant 6400R
Siebel Gatew ay Server
3%
200
4 x Proliant DL580
Siebel Application Server – End Users
13%
5,000
8 x Proliant BL40p
Siebel Application Server – End Users
11%
4,700
1 x Proliant DL580
Siebel Application Server – EAI HTTP Adapter+ WF
25%
2,200
1 x Proliant DL760
Siebel Application Server – EAI-MSMQ Adapter
21%
916
1 x Proliant BL20p
1 x Integrity rx5670
Siebel Application Server – AM
Microsoft SQL Server 2000 (64-bit)
3%
47%
80
13,300
12,000 User Benchmark on HP/Windows/SQL64

Concurrent Users
Workload
Avg Operation
Business
Num ber
Response Tim e to
Transactions
of Users
Load Runner (sec) Throughput / Hour
Projected
Transactions
8 Hour Day
Sales / Service Call Center
8,400
0.137
43,662
349,300
eChannel (PRM)
1,200
0.131
16,130
129,037
eSales
1,200
0.144
8,164
65,313
eService
1,200
0.162
15,462
123,694
N/A
83,418
667,344
12,000
 Totals
Server Component Throughput
Workload
Assignm ent Manager
Business
Transactions
Throughput / Hour
Projected
Transactions
8 Hour Day
62,012
496,096
EAI - HTTP Adapter
496,056
3,968,448
EAI - MQ Series Adapter
Workflow Manager
294,539
116,944
2,356,312
935,552
SQL64 on a 4x 1.5 GHz
Itanium2 HP Integrity
used 47% CPU and 13.3
GB memory proving
unprecedented
price/performance for
Siebel
Oracle 10K vs SQL Server 12K on the DB-tier
Oracle supported 10K users on rp8400 with 16x CPU 875Mhz with
Oracle 9.x/Hp-ux posting 35% CPU and 18.2GB memory.
MSSQL supported 12K users on rx5670 with 4x CPU 1.5Ghz with
sql2K/windows2003 posting 47% CPU and 13.3GB memory.
Result:
17.95% less CPU**
26% less memory
60% less cost
20% more users (12K vs 10K)
SQL Server 2000 64-bit did more with less.
*HP rx5670 is around $50K on HP web if you pay the full price.
*HP rp8400 base price $124K
** rp8400 16 CPU SpecInt 98.2, rx5600 4 CPU SpecInt 60
** (cont) (98.2 * 35%) = 34.37 , (60 * 47%) = 28.2
** (cont) 1 – (28.2/34.37) = 17.95%
** www.spec.org
** Scott Hall Slide
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Tuning Tools

Windows System Monitor – PERFMON.EXE
 System Monitor in the Performance console, started from Administrative
Tools

Query Analyzer – ISQLW.EXE
 Graphical showplan
 Statistics profile

Profiler- PROFILER.EXE
 Spot problematic queries
 Use the Tuning or Duration templates
 Monitor the overhead carefully on your system

Index Tuning Wizard – Particularly for EIM initial
Exploring I/O with System Monitor

Make sure you set DISKPERF –Y on command line to get counters dumped

Performance Object: Physical Disk

Counters:

%Disk Time

Number for entire logical drive should be less than 100%

Avg Disk Queue Length (system requests on avg. waiting for disk access)

Want 0

If it is above 2 (def. above 3), look into it

See if it’s sustained queueing or temporary

Avg. Disk Read/Write /sec (diff counters, and remember Logical vs. Physical)

Nice to have: 5 – 7 ms (might be optimistic)

Realistic (today’s technology): 20 – 25 ms on a moderately loaded system

Log device service write times should be below 20 ms

Technology dependent …

See BOL (index “Bottlenecks” then “Monitoring Disk Activity”) for some more tips
System Monitor

Useful Counters

Processor - % Processor Time

Physical Disk - %Disk Time, Avg. Disk Queue Length

Memory – Available MBytes

System – Context Switches / sec

SQL Server Locks – Lock Waits/sec, Number of Deadlocks/sec

SQLServer: Access Methods

Full Scans/sec, Page Splits/sec, Table Lock Escalation/sec

SQLServer: Buffer Manager

Buffer Cache Hit Ratio, Lazy Writes/sec, Page Reads/sec, Page
Writes/sec, ReadAhead Pages/sec

SQLServer: Databases - Transactions/sec

SQLServer: General Statistics - User Connections

Q150934 – How to Create a Performance Monitor Log
Profiler Terminology

Template



Trace


Limits the results (Equal, Not like, etc…)
Event Category


Captures data based upon selected events, data columns, and filters
Filter


Defines criteria for what to monitor
Saved in .tdf file
Defines the way events are grouped
Event

Action generated within SQL engine
Profiler

Use Built-in templates

Find the worst-performing queries

Filter by duration

Identify the cause of a deadlock

Monitor stored procedure performance

Audit activity – C2 audits

Reorder your output columns by Duration, CPU, read, writes,
textdata, etc.
Profiler in Production

Can be very CPU intensive
 My experience in production: 8x at 100%

Filter! Filter! Filter!

Let the computer tell you what’s going foul

To proactively find out what is going wrong
 Filter on Duration > 30,000ms
 Run for 24 hours
 This will show you all the poor running queries on your system.

Can’t make it run faster?
 Look at IO’s in loops or running all the time.
 Repack to higher fill factor.
 5 to 4 IO’s is a 20% increase.

McBath’s Oil Argument
 Do you notice your car running better when you change your oil? No.
But your car sure does.
 Make your database server run as efficient as possible. Makes it
more scalable. More with less.
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Siebel Database

Over 2,300 Tables

Many with over 120 columns per table

2,200 Clustered Indexes (ROW_ID)

10,500 Non-Clustered Indexes

Over Indexed, many NULL indexes

10,000+ Default Constraints

25,000+ Check Constraints

Very few Stored Procedures

Some Triggers

used by workflow / assignment manager
Siebel Logical Schema vs. Physical Schema

Siebel database schema is designed as a cross platform logical
schema, which is managed by Siebel Tools

The logical schema is translated to a physical schema by using
Siebel database utilities, such as DDLIMP

The logical schema may be altered and mapped to a compatible
physical data type depending on the DB platform and code page
Don’t Be Afraid to Add/Change Indexes…
 You almost always have to.
 Work closely with Siebel Expert Services
 Make sure your Siebel meta data is in sync with SQL
Server meta data… or bad things can happen next time
you DDLSYNC…
 See examples in the next slides
Reality (I)

Out of Box…

A Large Customer…

sp_helpindex EIM_CONTACT3

sp_helpindex EIM_CONTACT3

EIM_CONTACT3_T01

EIM_CONTACT3_T01

EIM_CONTACT3_T02

EIM_CONTACT3_U1

EIM_CONTACT3_T03

EIM_CONTACT3_T04

EIM_CONTACT3_T05

EIM_CONTACT3_T06

EIM_CONTACT3_T07

EIM_CONTACT3_T08

EIM_CONTACT3_T09

EIM_CONTACT3_T10

EIM_CONTACT3_T11

EIM_CONTACT3_T12

EIM_CONTACT3_T13

EIM_CONTACT3_T14

EIM_CONTACT3_T15

EIM_CONTACT3_T16

EIM_CONTACT3_T17

EIM_CONTACT3_T18

EIM_CONTACT3_T19

EIM_CONTACT3_T20

EIM_CONTACT3_U1
Reality (II)

S_CONTACT_EI

S_CONTACT_F10

S_CONTACT_F11

S_CONTACT_F12

S_CONTACT_F13

S_CONTACT_F15

S_CONTACT_F2

S_CONTACT_F3

S_CONTACT_F4

S_CONTACT_F5

S_CONTACT_F6

S_CONTACT_F7

S_CONTACT_F8

S_CONTACT_II

S_CONTACT_M1

S_CONTACT_M11

S_CONTACT_M12

S_CONTACT_M13

S_CONTACT_M14

S_CONTACT_M15

S_CONTACT_M16

S_CONTACT_M17

S_CONTACT_M18

S_CONTACT_M19

S_CONTACT_M2

S_CONTACT_M20

S_CONTACT_M21

S_CONTACT_M22

S_CONTACT_M3

S_CONTACT_M4

S_CONTACT_M6

S_CONTACT_M8

S_CONTACT_M9

S_CONTACT_P1

S_CONTACT_U1

S_CONTACT_U2

S_CONTACT_V1

S_CONTACT_V2

S_CONTACT_V3

S_CONTACT_V5

S_CONTACT_EI

S_CONTACT_F6_X

S_CONTACT_II

S_CONTACT_M1

S_CONTACT_M50

S_CONTACT_M8

S_CONTACT_ML1_X

S_CONTACT_ML2_X

S_CONTACT_ML3_X

S_CONTACT_ML4_X

S_CONTACT_ML5_X

S_CONTACT_ML6_X

S_CONTACT_P1

S_CONTACT_PREM01_X

S_CONTACT_PREM02_X

S_CONTACT_U1

S_CONTACT_U2

S_CONTACT_V3

Indexes in RED were custom.
Poor Indexes
 Get rid of 100% NULL indexes
 Cost a lot on INSERTS/UPDATES/DELETES, ex. EIM
 Cost disk space
 Cost tape space when you back them up
 Only time one might be used: on an aggregate. It’s cheaper
than a full scan. Rare, though.
 Stored Procedure below will examine the indexes on the
top 100 tables for indexes that probably will not be used.
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Siebel OM Query Execution and Fetching Mechanism
 20 tables in a join…
 Siebel OM uses Server side API cursors
 For List applet functionality i.e. to maintain
user state and support pending result sets
 To support multiple active statements per
connections
 Fast Forward cursor with auto-fetch or
Dynamic cursor when accessing text columns
 Sometimes there is an implicit conversions to
Keyset (order by not covered by index )
 Average fetch size is 3 or 4 rows – this is
computed by dividing the ODBC buffer size by
row size
 Siebel uses ODBC Prepare/Execute
 Example: Select * from table where x = ?
 What it looks like…
SQL Fetch Mechanism
sp_cursorprepe
x (….., 3)
Siebel
OM
3 rows
sp_cursorfetch
(…,3)
3 rows
c1
c2
c3
c4
c5
a
b
c
d
e
f
g
h
i
j
a
b
c
d
e
f
g
h
i
j
a
b
c
d
e
f
g
h
i
j
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Typical Siebel OM Query
declare @P1 int
set @P1=-1
declare @P2 int
set @P2=0
declare @P3 int
set @P3=28688  Fast Forward, Parameterized, Auto Fetch, Auto Close (undocumented and subject to change)
declare @P4 int
set @P4=8193
declare @P5 int
set @P5=10
exec sp_cursorprepexec @P1 output, @P2 output, N'',N'
SELECT
T1.LAST_UPD_BY,
T1.ROW_ID,
T18.PRTNR_TYPE,
T13.CREATED_BY,
T2.ASGN_USR_EXCLD_FLG,
.
.
.
T2.PAR_OU_ID
FROM
dbo.S_PARTY T1
INNER JOIN dbo.S_ORG_EXT T2 ON T1.ROW_ID = T2.PAR_ROW_ID
INNER JOIN dbo.S_ACCNT_POSTN T3 ON T2.PR_POSTN_ID = T3.POSITION_ID AND T2.ROW_ID = T3.OU_EXT_ID
INNER JOIN dbo.S_PARTY T4 ON T3.POSITION_ID = T4.ROW_ID
LEFT OUTER JOIN dbo.S_ORG_EXT T5 ON T2.PAR_OU_ID = T5.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_PRI_LST T6 ON T2.CURR_PRI_LST_ID = T6.ROW_ID
LEFT OUTER JOIN dbo.S_POSTN T7 ON T2.PR_MGR_POSTN_ID = T7.ROW_ID
LEFT OUTER JOIN dbo.S_USER T8 ON T7.PR_EMP_ID = T8.ROW_ID
LEFT OUTER JOIN dbo.S_ORG_EXT T9 ON T2.PAR_BU_ID = T9.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_ORG_EXT T10 ON T1.PAR_PARTY_ID = T10.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_ORG_PRTNR T11 ON T1.ROW_ID = T11.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_ORG_EXT_SS T12 ON T1.ROW_ID = T12.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_BU T13 ON T1.ROW_ID = T13.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_OU_PRTNR_TIER T14 ON T2.PR_PRTNR_TIER_ID = T14.ROW_ID
LEFT OUTER JOIN dbo.S_ASGN_GRP T15 ON T2.PR_TERR_ID = T15.ROW_ID
LEFT OUTER JOIN dbo.S_INDUST T16 ON T2.PR_INDUST_ID = T16.ROW_ID
LEFT OUTER JOIN dbo.S_ADDR_ORG T17 ON T2.PR_ADDR_ID = T17.ROW_ID
LEFT OUTER JOIN dbo.S_OU_PRTNR_TYPE T18 ON T2.PR_PRTNR_TYPE_ID = T18.ROW_ID
LEFT OUTER JOIN dbo.S_POSTN T19 ON T3.POSITION_ID = T19.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_USER T20 ON T19.PR_EMP_ID = T20.PAR_ROW_ID
LEFT OUTER JOIN dbo.S_ORG_SYN T21 ON T2.PR_SYN_ID = T21.ROW_ID
LEFT OUTER JOIN dbo.S_ORG_BU T22 ON T2.BU_ID = T22.BU_ID AND T2.ROW_ID = T22.ORG_ID
LEFT OUTER JOIN dbo.S_PARTY T23 ON T22.BU_ID = T23.ROW_ID
LEFT OUTER JOIN dbo.S_ORG_EXT T24 ON T22.BU_ID = T24.PAR_ROW_ID
WHERE
((T2.PRTNR_FLG != ''N'') AND
((T13.BU_FLG = ''N'' OR T13.BU_FLG IS NULL) AND T2.PRTNR_FLG = ''Y''))
OPTION(FAST 40)', @P3 output, @P4 output, @P5 output
Build plan and return
40 rows ASAP
Why do I get a different query plan in the Query Analyzer ?
 Bind value (Prepare Execute model)
 Hard Coding Values instead of binding at Run Time
 Cursor (SQL Server API cursor)
 Not putting the “ODBC Wrapper”
 SQL hint (Fast 40)
 Not including compiler options
 Text column
 Implicit Cursor Conversion
 Table Spools in one plan, but not the other
 Capture on Implicit Cursor Event in Profiler
 Also capture on “integer data” column
(N)TEXT Columns

(N)TEXT column may cause performance problems

In the Siebel database schema


A logical TEXT data type is always translated to a physical (N)TEXT column
A VARCHAR data type can be translated to either a (N)VARCHAR column or a
(N)TEXT column
 VARCHAR(2000+) is translated to a (N)TEXT column
One size fits all: Implicit Cursor Conversions
 One type of cursor is requested, but it cannot be fulfilled in
it’s native call.
 Rather than fail, SQL Server converts internally.
 Performance problems.
 For example, Siebel uses an “option fast 40”
 Fast forward, read only requested with an ORDER BY, yet
no index on the WHERE clause that is ordered. SQL Server
converts to a KEYSET cursor which spools off to TEMPDB
for the sort.
 Fix: make an index that matches the ORDER BY.
 KEYSET conversion goes away.
 SQL Profiler: Event -> Cursors ->
CursorsImplicitConversion
Profiler Properties
ODBC implicit cursor conversions: BOL
Query Repro: Quick and Dirty
 Capture on RPC: Starting Event on Profiler
 Cut and paste it into Query Analyzer
 99% of the time it will give you the same plan that is
coming out of Siebel.
 DON’T: Spool Siebel out at the client, hard code the values
and put into Query Analyzer.
 Probably won’t work
 For example, it won’t have the OPTION FAST 40
How to make the Query in QA

print 'declaring variables'

declare @P1 int

declare @P5 int

declare @P6 int

set @P1=NULL

-- set @P5=28688

-- SCROLLOPT 28676 = 16384 (AutoClose) + 8192 (AutoFetch) + 4096 (Parameterized) + 4 (Forward Only)

set @P5=28676

set @P6=8193

print 'running sp_cursorprepare'

exec sp_cursorprepare @P1 output


, N'@P1 varchar(30)'
--
, N'SELECT * FROM authors WHERE au_lname like @P1 OPTION (FAST 1)'

, N'SELECT * FROM authors WHERE au_lname like @P1'

,1

, @P5 output

, @P6 output

print 'declaring more variables'

declare @P2 int

declare @P3 int

declare @P4 int

set @P3 = 1

set @P2=NULL

set @P3=24592

-- SCROLLOPT - 24592 = 16384 (AutoClose) + 8192 (AutoFetch) + 4 (Forward Only)

set @P4=8193

set @P5=15

print 'executing cursor'

exec sp_cursorexecute @P1, @P2 output, @P3 output, @P4 output, @P5 output, 'R%'

print 'select the results from the cusor execute'

select @P1,@P2,@P3,@P4, @P5, @P6
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Siebel and Update Statistics

Problem:




Siebel queries join a lot of tables
A lot of the tables joined might be smaller than the threshold to execute an
automatic update statistics
Result: Bad plans for the join
Stale statistics and EIM


A Plan that tips over. 98 jobs run in 5 minutes. 2 run in 1 hour or… a lot longer.
EIM running faster than Auto Update Stats can kick off.

Solution run a manual update statistics on those tables

Check rowmodctr in sysindexes for indid=1

See Appendix for code on auto update stats
Example of Stale Statistics

set statistics io on

Table ‘S_CONTACT'. Scan count 0, logical reads 0, physical reads 0,
read-ahead reads 0.

Table ‘S_OPPTY'. Scan count 4382, logical reads 21439, physical
reads 650, read-ahead reads 877.

Table ‘S_PARTY'. Scan count 2, logical reads 17573, physical reads
199, read-ahead reads 0.

After UPDATE STATISTICS with sample of 10%:

Table ‘S_CONTACT'. Scan count 0, logical reads 0, physical reads 0,
read-ahead reads 0.

Table ‘S_OPPTY'. Scan count 192, logical reads 1440, physical
reads 0, read-ahead reads 11.

Table ‘S_PARTY'. Scan count 4, logical reads 1507, physical reads 3,
read-ahead reads 59.
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Why Wait Types?
 Wait types will help define where your bottleneck is.
 They are seen in the master..sysprocesses table in a
column called “waittype”.
 select waittype, * from master..sysprocesses
 There are all kinds of waittypes. For example, blocking due
to database locks, network io, disk queueing, etc…
 The key to solving throughput is understanding what’s
damming up the river.
Wait Types
 If a thread goes into a sleep status, a wait state is set.
 The wait state is contained in master..sysprocesses in the
columns waittype, and lastwaittype.

Lastwaittype is a character description of the last wait
state for this thread. It is not reset until another wait state
occurs.
 Waittype is a varbinary wait state that is the current wait
state. A wait time of 0 means the thread is currently
running.
 See SQL_PERF.DOC for detailed information:
Track_waitstats stored procedure
 Track_waitstats is a stored procedure that will capture
waitstats from DBCC SQLPERF, and provide a ranking of
descending order based on percentage.

This is useful in identifying the greatest opportunites for
performance improvements.
 See Appendix for Stored Procedure on Perf
 See the sample output below:
Sample Output
Commentary

The above sample shows the majority share of wait time, 48%,
being due to network IO waits. Improving network IO is the
single largest opportunity for improving application performance.

Other lesser opportunities in the above example include
LCK_M_X (exlusive locks) and WRITELOG (transaction log).
Exclusive lock waits account for almost 13% of total wait time.
An examination of transaction management may offer clues as to
whether improvements can be made here.

WRITELOG means threads are waiting for physical writes to
complete to the transaction log. Given the 11% writelog waits, a
further analysis of PERFMON disk queues for the transaction log
will confirm whether the IO capacity of the transaction log drives
have trouble keeping up with write requests as shown by steady
and high disk queues.
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Locating the Query Using the Most CPU

Perfmon

Counters:
 Threads:



% Processor Time
ID Process (NT PID)
ID Thread (KPID)
 select spid
 from master..sysprocesses
 where kpid = <ID Thread>
 Select all SQLSERVR/0 to SQLSERVR/99
 DBCC INPUTBUFFER(spid)
Quick Estimate for Long Running Queries





select datediff(mi, last_batch,getdate())'minutes',spid, waittype, cpu,
physical_io, convert(char(15),hostname), convert(char(15),program_name),
convert(char(20),getdate()),spid, last_batch, cmd
from master..sysprocesses
where spid > 50 and
cmd not like '%WAIT%' and
datediff(mi, last_batch,getdate()) > 1

order by last_batch
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
EIM & Data Loading
 I’m short changing you… I have a 2 hour presentation just
on this subject!
 The following are just some “good ideas” on generic
loading of data.
 Attached is a 50 ppt presentation on EIM and SQL Server
 Please contact me off line for information on this topic.
Glad to work with you on it.
Cursors, DTS, Loading Data and You
 Cursors are the ISAM / Dbase way of thinking about a
problem.
 Think in set wise solutions
 Performance is drastically improved:
 4:28 minutes to 23 seconds
 Show example with “set wise” and “cursor based”
 Compare Updateable vs. Fast Forward/Read Only
 Fair Warning: DTS
 DTS works great. Dev types tend to badly abuse it.
 “Let’s draw a cool picture…”
 Run it over a network
 Want to loop one row at a time
 ActiveX controls
 Rewrote a cleansing process from 15 minutes to 20 seconds by
eliminating ActiveX
 See attached presentation on high perf data loading.
Cursor

DECLARE scrub_cursor CURSOR <FAST_FORWARD> FOR

SELECT y

FROM table_x <nolock>

order by y

-- 4:23 minutes

-- normal:


------------------------------------------------------

2003-08-06 09:48:26.560

------------------------------------------------------

2003-08-06 09:52:49.230

-- 3:50 minutes

-- fast forward:


------------------------------------------------------

2003-08-06 09:56:16.140

------------------------------------------------------

2003-08-06 10:00:05.780
Set Wise

Just as much “work” for SQL Server to crunch 1 row as 10K rows…


select getdate()
go



update table_x
set ROW_ID = '0'
go





select getdate()
go
-- 26 seconds
-----------------------------------------------------2003-08-06 09:44:14.263



(943093 row(s) affected)
-----------------------------------------------------2003-08-06 09:44:40.030
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Balanced System Design

All systems have a bottleneck…
 But a balanced system tries to maximize all system
components
 And pushes the bottleneck to the most “expensive”
component

CPU is piece to maximize
 Costs $5000 each
 Hard to add more CPUs without changing whole hardware
architecture
 System bus not fast enough
 Other pieces more readily expandable
I/O Subsystem (I)

Disks becoming bigger and bigger in terms of volume

Access speed does not increase at the same rate

Problem: Disk access speed essential for I/O performance

Don’t maximize on volume but on # of spindles

Typical disk performance
I/O sec
8K
64K
Sequential I/O
150
1.2 MB/sec
9.4 MB/sec
Random I/O
100
0.8 MB /sec
6.3 MB/sec
I/O Subsystem (II)

Recommendation since ever:
 Log File separated from Data Files on RAID1 or RAID1+0
 Data Files minimum protection RAID5

RAID5 has severe disadvantages in writing blocks due to the fact
that checksum for block has to be read, calculated and written
down

Especially for applications with time sensitive import workload
(like nightly EIM Extracts) RAID1+0 for data files are
recommended
I/O Subsystem (III)

SAN storage backend
 Usually huge caches
 Own methods of building volumes to optimize data
distribution over a high number of disks

Issues:
 Caches do not speed up all kind of operations. Typical
example: Backup or checkdb
 Highly complex data distribution on this hardware might hide
the fact that portions of Data files and Log files end up on the
same spindle
 Advertisement might indicate that a physical separation of
Data files and Log files are not necessary anymore
 Still insist on physical separation of Date file and Log File
I/O Subsystem (IV)

Fiber-Channel and SCSI adapter have bandwidth limitations as
well.
 Some are >100MB/sec according to spec
 These are max. values with burst and other special cases. In
normal operations bottleneck appear way earlier

PCI-Bus has limitations as well
 E.g. Profusion 8way architecture. Not every PCI slot has full
throughput
I/O Subsystem (V)

How to detect I/O bottlenecks
 Avg. Disk Queue/sec with Perfmonitor
 Avg. sec/transfer with Perfmonitor

Avg. sec/transfer should not be beyond 20ms for the data
partitions and less than 10ms for transaction log partitions

Attention: Due to SQL Server’s architecture the maximum queue
for a dedicated transaction log partition (recommended) will not
be higher than 3.
 Therefore Avg. Disk Queue/sec is not very helpful for
analyzing performance issue on the transaction log partition
 Log stalls hold up transaction commit and can lead to
significant concurrency issues
I/O subsystem (VI)

Select * from ::fn_virtualfilestats(-1,-1)

Show stall time per I/O request on a per file basis
 Select DbId, FileId, IoStallMS/(NumberReads+NumberWrites)
‘ms/io’ from ::fn_virtualfilestats(-1,-1)
 Limits for data files the same. Due to different measurement
value for log can be a little higher than with perfmon
What the numbers say…
 Stall time per I/O
 This should gives a good idea on I/O
 Disadvantage of this solution is that the counters
are not resetable.
 What happens then over time is that the
immediate impact of some performance issues
disapears in the clutter
A more in depth look…
 SQL Server Profiler Counters:
 Physical disk avg disk sec/read
 Physical disk avg disk sec/write
 Takes into account disk queues
 If not queuing then you’d normally expect to see reads
taking 4-10ms as low as 1ms (w/cache).

If you know (in the absence of queuing) a read should take 6ms
and you are seeing an average of say 15ms, then you have can
see the effect of sustained queuing. Thus you have an IO
bottleneck.
Memory (I)

With Windows 2000 Advanced Server up to 8GB real memory can
be addressed (/PAE entry in boot.ini)

Still one single process is limited on max 3GB

But all processes can share the 8GB real memory

SQL Server 2000 Enterprise Edition can leverage up to 8GB as
well using AWE
Memory (II)
 Does it make sense to use Datacenter Edition to leverage
more than 16GB memory for SQL Server 2000
 Typically not under a typical workload
 Only SQL Server data pages can reside outside the
virtual address space of 3GB
 Lock structures, query plans, buffer headers and other
highly frequented structures can not use AWE memory
 Recommendation: Usage of AWE up to 8GB by SQL Server
is fine and can show great performance improvement
 Having /PAE enabled the system is not able to read or
write 64K Tape I/O blocks
 Backups performed on a system without /PAE
enabled are not readable on a system with /PAE
 Online backup against tape could take a little longer
File Management

Everlasting question: How many data files should be used?
 Ideally between 1 and 3 per disk array
 No need to restrict file size for performance reasons
 Minimum # of data files around also dependent on # CPUs
 Ratio of .25 - .5 times # of CPUs is sufficient to guarantee
good performance
 Grow files manually to guarantee optimal proportional fill
Sample SQL Disk Configuration
SQL Server
Fibre Controller
Fibre Controller
Fibre Switch Fibre Switch
SCSI
SCSI
SCSI
LTO

12 100gb logical disks
(Raid 0+1)

1 SQL file per disk (H: S:)

3 Drive LTO Array



StorageWorks disk subsystem
LTO
1.4 TB Backed up in > 3
hours (130 MB/Sec)

Faster than backing up
to Disk
Online Backup
No user performance
impact
Network Performance
 Load all the data locally
 Named Pipes vs TCP/IP
 Named Pipes if on the same server will run faster
 Can control through the SQL Client Network utility
Agenda
Introduction
Benchmarking
Tools & Monitoring
Siebel Database
Siebel Queries
Siebel Query Repro
Siebel & Statistics
Wait Types
Locating Long Running Queries
Optimizing Data Loading
Balanced System Design
- Network Performance
- IO
- Memory
- File Management
Summary
Summary of Tuning
SQL 2000 Tuning Specifics

Disable all unnecessary Siebel component groups to reduce system resources

Data file placement – Data, Transaction Log, Tempdb





Keep on separate spindles
Pre-allocate space for DB and Tempdb
Autogrow on
Physically isolate SQL Transaction Log on mirrored drive with no other I/O
Raid: use many spindles to allow parallel retrieval of data

Use Siebel database connection pooling to reduce the memory footprint on the databas
server

Indexing


Clustered index scans vs. Clustered index seeks
Use /3GB in windows boot.ini on SQL32 for >5K Siebel users
Common Mistakes…

Not believing in Dynamic Control



Believing too much in Dynamic Control



Not paying attention to indexing
Not paying attention to resources
Lack of understanding



Index hints
Not utilizing autostats
Recovery mechanisms
Fragmentation
Inappropriate use of Cursors during data loading – non Siebel processes
SQL 2000 Tuning Specifics

Maintain accurate statistics
 AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS ON
 Manually Update stats after large data modifications
 DBCC SHOW_STATISTICS

Isolate specific tables / indexes into SQL Filegroups
 Siebel lists candidate tables in documentation (eg. S_DOCK_TXN_LOG)
 Separate EIM interface tables from Base tables

Allow dynamic memory growth

Index fragmentation
 DBCC SHOWCONTIG
 DBCC INDEXDEFRAG – online operation

Tune SQL
 Query plan analysis
 Cursor implicit conversions
SQL Server tuning guidelines
 Index fragmentation
 DBCC SHOWCONTIG
 DBCC INDEXDEFRAG – online operation
 DBCC DBREINDEX – offline operation (ex. can block)
 Windows System Monitor
 System Monitor in the Performance console, started from Administrative
Tools
 Profiler
 Spot problematic queries
 Use the Tuning or Duration templates
 Monitor the overhead carefully on your system
 Graphical showplan in Query Analyzer
 Index Tuning Wizard – Particularly for EIM initial
SQL Server tuning guidelines
 Data file placement – Data, Transaction Log, Tempdb




Keep on separate spindles
Pre-allocate space
Autogrow off or very small increment on Transaction Log and Tempdb
Physically isolate SQL Transaction Log on mirrored drive with no other I/O
 Indexing
 too many
 too few
 Clustered index scans vs. Clustered index seeks
 Maintain accurate statistics
 AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS
ON
 Manually Update stats after large data modifications
 Isolate specific tables / indexes into SQL Filegroups
 Siebel lists candidate tables in documentation (eg. S_DOCK_TXN_LOG)
 Separate EIM interface tables from Base tables
 Allow dynamic memory growth
SQL Server tuning – Actual EIM case
 Problem: EIM initial processing 30,000 – 40,000 rows per hour
 Final Results: EIM initial processing 1,300,000 rows per hour
 Process: Standard Siebel Expert Services methodology
 Tools:
 Siebel SQL Tracing for EIM to show 10 most expensive queries
 Window System Monitor to measure I/O, CPU and memory
 SQL Server Profiler to capture database events
 Steps:







Spot problematic queries – long running insert
Memory, SQL settings all fine, I/O was poor due to HW problems
Use Query Analyzer (Show Execution Plan) to determine whether additional index
will help – in this case: No
Minimize I/O by dropping indexes on Base Tables (S_CONTACT) – 400,000+
Separate data into batches by key ranges to avoid deadlocks and run 4 EIM
processes in parallel – 1.3 Million rows / hour
Recreate indexes, update stats, defrag table and indexes
Investigate I/O subsystem Hardware / SAN
Thanks!
 Thank you for attending!
 Frank McBath
 [email protected]
Appendix Matter
Performance Isolation and Resolution
Poor Database
Response Time
Disk Subsystem
Architecture:
Controllers, Logs,
Data Files,
TempDB
Spread Out The
Load
Type of
Problem
Watch Perfmon
High CPU
Utilization
Query
Perfmon: THREAD
Counters
High IO
STATISTICS
IO
Look at Contig
Space
DBCC
SHOWCONTIG
Defrag: Space, Fill
Factor
DBCC REINDEX/
INDEXDEFRAG/
SELECT INTO
Unpredictable
Plan
Look At Plan
SHOWPLAN
With RECOMPILE
Book Mark Lookup
STATISTICS
PROFILE
Look At Statistics
DBCC
SHOW_STATISTI
CS
Rearrange the
Columns
DBCC
SHOW_STATISTI
CS
Hard Code Hint
(INDEX = )
WITH
RECOMPILE
Covered Index
UPDATE STATS
with
HigherSampling
Frequency
Create New Index
Using Correct
Index
Selective
Enough?
“BOE” method
Kill Run Away Job/
SPID
Scope Query
Results Down:
Tighten up
WHERE
Supported Platforms: OS/DB
Current Siebel 7.5 Platform Support
Database Server
Platforms
Microsoft SQL Server 2000
Microsoft SQL Server 2000 (64-bit)
eBusiness Server
Platforms
Microsoft Windows NT 4.0
Microsoft Windows 2000 Server / Advanced Server
Microsoft Windows 2000 Datacenter
Cluster Software
Microsoft Clustering Service for Windows NT 4.0
Microsoft Clustering Service for Windows 2000
Web Servers
Microsoft IIS 4.0
Microsoft IIS 5.0
HI Mode Applications
Browsers: Microsoft IE 5.5, Microsoft IE 6.0
Operating Systems: Windows NT / 2000 / XP
SI Mode Applications
Browsers: Microsoft IE 5.01 / 5.5 / 6.0, MSN 8.0
Operating Systems: Windows 95 / 98 / ME / NT 4.0 / 2000 / XP
New platforms or versions are highlighted in RED
Planned Siebel 7.7 Platform Support – Q1, 2004
Database Server
Platforms
Microsoft SQL Server 2000
Microsoft SQL Server 2000 (64-bit)
eBusiness Server
Platforms
Microsoft Windows 2000 Server / Advanced Server /
Datacenter
Microsoft Windows 2003 Server / Advanced Server /
Datacenter
Cluster Software
Microsoft Clustering Service for Windows 2000
Microsoft Clustering Service for Windows 2003
Web Servers
Microsoft IIS 5.0
Microsoft IIS 6.0
HI Mode Applications
Browsers: Microsoft IE 5.5, Microsoft IE 6.0
Operating Systems: Windows NT / 2000 / XP
SI Mode Applications
Browsers: Microsoft IE 5.01 / 5.5 / 6.0, MSN 8.0
Operating Systems: Windows 98 / ME / NT 4.0 / 2000 / XP
New platforms or versions are highlighted in RED
Performance & Scalability
Improved
Performance
Improved
Scalability
Up to 64-way SMP
Up to 512 GB RAM
NUMA Optimization
Intel Hyper-Threading
Technology
Native 64-bit support (Itanium)
Hardware vendor support
• 64-way HP SuperDome
Support for New
Scalable Hardware
• 32-way NEC Express5800
• 32-way Unisys ES7000 Orion 130
• 16-way IBM eServer xSeries 440
Additional resources
 Siebel Supportweb
 http://ebusiness.siebel.com/supportweb
 Periodically check for new Tech Alerts and Updates
 SQL Server Books Online
 KB Articles:
 298475 : Information Required to Successfully Troubleshoot Application
Performance Issues
 243589 : HOW TO: Troubleshoot Slow-Running Queries on SQL Server 7.0 or
Later
 SQL Server User groups
 Books:
 SQL Server 2000 Performance Tuning, Microsoft Press
• Good sections on RAID, Capacity Planning, System Tuning
 Inside SQL Server 2000, Microsoft Press
• The one book to have on SQL Server
Creating server based trace
DECLARE @RC int, @TraceID int, @on BIT
EXEC @rc = sp_trace_create @TraceID output, 0, N'C:\test7'
SELECT RC = @RC, TraceID = @TraceID
SELECT @on = 1
exec sp_trace_setevent @TraceID, 10, 1, @on
exec sp_trace_setevent @TraceID, 10, 2, @on
exec sp_trace_setevent @TraceID, 10, 3, @on
exec sp_trace_setevent @TraceID, 10, 6, @on
exec sp_trace_setevent @TraceID, 10, 7, @on
exec sp_trace_setevent @TraceID, 10, 8, @on
exec sp_trace_setevent @TraceID, 10, 9, @on
exec sp_trace_setevent @TraceID, 10, 10, @on
-- Set the Filter
EXEC sp_trace_setfilter 1, 10, 0, 6, N'MS%‘
-- Start Trace (status 1 = start)
exec @RC = sp_trace_setstatus @TraceID, 1
GO
Additional resources

Siebel Supportweb
 http://ebusiness.siebel.com/supportweb
 Periodically check for new Tech Alerts and Updates

KB Articles:
 298475 : Information Required to Successfully Troubleshoot Application Performance Issues
 243589 : HOW TO: Troubleshoot Slow-Running Queries on SQL Server 7.0 or Later

SQL Server User groups

Books:
 SQL Server Books Online – look here first
 SQL Server High Availability, Microsoft Press
 SQL Server 2000 Performance Tuning, Microsoft Press
• Good sections on RAID, Capacity Planning, System Tuning
 Inside SQL Server 2000, Microsoft Press
• The one book to have on SQL Server
Siebel 7.5 - MCI
Proof of Concept / Implementation

Test demonstrated scalability of SQL Server 2000


Large anticipated customer workload




500,000 accounts
8,000 Concurrent Users
Sub-second response times for majority of transactions
Real world analysis




Tested Siebel 7.5 implementation of Sales Force Automation
Provided MCI with validation of proposed architecture for their Sales Force Transformation (SFT) Program
Real world analysis using MCI’s hardware configuration and existing Siebel data
Initial Scalability testing at the Accenture High Performance Scalability Lab in Cincinnati, OH
From MCI’s John Heveran’s presentation at Siebel user week 2003 @ San Diego
Siebel customers realize significant On-Going TCO Benefits
Three year total cost of ownership comparison on Siebel deployments
Resource
SQL Server 2000
Competing
Product
Savings
% Difference
% Savings
Hardware
$47,602
$109,616
$62,014
57%
8%
Software
$103,596
$160,000
$55,404
35%
7%
Hardware Maintenance
$7,777
$21,652
$13,875
64%
2%
Software Maintenance
$58,161
$79,049
$20,888
26%
3%
$791,704
$1,369,040
$577,336
42%
76%
$2,279
$31,530
$29,251
93%
4%
$1,011,119
$1,770,887
$758,768
43%
100%
Ongoing Operations
Support Activities
Training
Total Cost of Ownership
http://www.microsoft.com/sql/evaluation/compare/migratingSiebelCRM.asp
 NerveWire study reveals a three-year ROI of 248% as a result of migrating Siebel CRM applications to
SQL Server2000 on Windows 2000 from a competitive RDBMS on Unix.
 A company with sales of $2B/Yr. and a gross profit margin of 5% would need to increase sales by $20MM
over 3 years to accomplish the same results!
Siebel Scalability On Available Platforms
Workload
(User Type)
AIX & DB2
Sales / Service Call Center
Avg Operation Response Tim e to
Load Runner (sec)
Num ber of Users
W2K &
SQL2K
HP-UX
AIX & DB2
W2K &
SQL2K
HP-UX
Business Transactions
Throughput / Hour
AIX & DB2
W2K &
SQL2K
HP-UX
20,000
20,000
22,400
0.148
0.295
0.116
122,041
121,425
116,571
eChannel (PRM)
4,000
4,000
3,200
0.182
0.185
0.212
27,615
27,619
42,890
eSales
3,000
3,000
3,200
0.233
0.207
0.242
17,134
17,157
21,703
eService
3,000
3,000
3,200
0.196
0.147
0.228
40,455
40,521
41,148
30,000
30,000
32,000
207,245
206,722
222,312
Totals
Workload
(Background Processing)
Business Transactions
Throughput / Hour
AIX & DB2
Assignm ent Manager
W2K &
SQL2K
HP-UX
38,599
37,693
22,817
EAI - HTTP Adapter
746,676
854,557
770,905
EAI - MQ Series Adapter
Workflow Manager
545,472
96,299
728,745
97,585
540,845
-
-
-
Metric
DB Grow th (Proj GB/Month)
AIX & DB2
W2K &
SQL2K
250.00
290.00
DB Mem ory Used (GB)
18.10
25.40
31.10
Database Connections
Web Server kbps per user
1,818
3,302
1,661
6.50
0.54
4.50
60,244
Note: 30,000 user tests are based on Siebel 7.0.3 and 32,000
test is based on 7.5.2; transaction mix is different between
Siebel 7,0.3 and 7.5.2 test suites.
N/A
Resource Utilization by 30,000 and 32,000 Concurrent Users Test
Node
30,000 Users
CPU
84%
0.74 GB
11%
0.59 GB
70%
14.82 GB
Siebel Servers (AM / EAI / WF)
2 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM
85%
0.54 GB
46%
1.054 GB
5 x HP rp5470 /w 4 875 MHZ & 16GB RAM
37%
0.816 GB
70%
0.070 GB
4 x HP rp8400 /w 16 x 875MHz & 64 GB RAM
82%
17.55 GB
1 x HP Superdome /w 32 x 875MHz & 128GB RAM
81%
34.70 GB
82%
2.90 GB
62%
31.10 GB
Web Servers (EAI HTTP Requests)
Siebel Servers (Object Managers)
Siebel Servers (AM / EAI / WF)
Database Server - IBM DB2 v7.2
1 x IBM p690 /w 32xRS64-IV 1.3GHz & 128GB RAM
5 x HP rp5470 /w 4 750 MHZ & 16GB RAM
1 x HP rp2470 /w 2 750 MHZ & 8GB RAM
Siebel Servers (Object Managers)
5 x IBM p680 /w 24xRS64-IV 600MHz & 64GB RAM
Mem
Web Servers (User Requests)
Web Servers (EAI HTTP Requests)
1 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM
32,000 Users
CPU
Mem
Web Servers (User Requests)
3 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM
Node
23%
18.10 GB
1 x HP rp8400 /w 16 x 875MHz & 64GB RAM
Database Server - Oracle 9.2.0.2
1 x HP Superdome /w 16 x 875MHz & 64GB RAM
Node
30,000 Users
CPU
Mem
56%
0.184 GB
39%
0.035 GB
48%
3.132 GB
19%
1.805 GB
2 x Unisys ES2081 /w 8 x PIII 700MHz (EAI HTTP/MQ Series)
57%
0.810 GB
Web Servers (User Requests)
8 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM
Web Servers (EAI HTTP Requests)
1 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM
Siebel Servers (Object Managers)
35 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM
Siebel Servers (AM / EAI / WF)
2 x Unisys ES2041 /w 4 x PIII 700MHz (AM / WF)
Database Server - MS SQL Server 2000 64-bit
1 x Unisys ES7000 Orion 130 /w 16 x Itanium 2 1GHz & 6467%
GB RAM
25.74 GB
Server consolidation / Mainframe class reliability
Unisys ES7000 Siebel “In-a-Box”
Typical Large Implementation
Web Servers
Application Servers
Web Servers
Siebel 7
Application Server
and
SQL
Database Server
Database
Server
Routers
Routers
Routers
Unisys ES7000
Unisys “In-a-Box” Siebel 5,000 user PSPP Benchmark new!
Monitoring Stored Procedure

create proc track_waitstats (@num_times int=5,@delaymin int=1)

as

--
@num_times is the number of times to capture waitstats, default is 5 times

--
default delay interval is 1 minute

--
create waitstats table if it doesn't exist, otherwise truncate

set nocount on

if not exists (select 1 from sysobjects where name = 'waitstats')

create table waitstats ([Wait Type] varchar(80),

Requests numeric(20,1),

[Wait Time] numeric (20,1),

[Signal Wait Time] numeric(20,1),

now datetime default getdate())

else

dbcc sqlperf (waitstats,clear) -- clear out waitstats

declare @i int,@delay varchar(8),@now datetime, @totalwait numeric(20,1),@endtime datetime,@begintime datetime

select @i = 1

select @delay='00:' + right('0'+convert(varchar(2),@delaymin),2) + ':00'

while (@i <= @num_times)

begin

truncate table waitstats
insert into waitstats ([Wait Type], Requests, [Wait Time],[Signal Wait Time])

exec ('dbcc sqlperf(waitstats)')

select @i = @i + 1

waitfor delay @delay

end

select @now=max(now),@begintime=min(now),@endtime=max(now),
@totalwait=max([wait time])

from waitstats where [wait type] = 'Total'

--- subtract waitfor, sleep, and resource_queue from Total

select @totalwait=@totalwait - sum([wait time]) from waitstats

where [wait type] in ('Waitfor','Sleep','Resource_Queue') and now = @now

-- insert adjusted totals, rank by percentage descending

insert into waitstats select '***total***',0,@totalwait,@totalwait,@now

select 'start, end, duration'='start: ' + convert(varchar(20),@begintime,20)
+ ' end: ' + convert(varchar(20),@endtime,20) + ' duration (minutes): ' +
convert(varchar(10), datediff(mi,@begintime,@endtime))

select [wait type],[wait time],percentage=cast (100*[wait time]/@totalwait
as numeric(20,1))

from waitstats

where [wait type] not in ('waitfor','sleep','resource_queue','total')

and now = @now

order by percentage desc
Siebel and Update Statistics Code
declare @tname sysname, @execstmt varchar(128)
set nocount on
declare c1 cursor for select object_name(id) from sysindexes
where indid=1 and id> 100 and rows<500 and rowmodctr >0
open c1
fetch c1 into @tname
while @@fetch_status<>-1
begin
set @execstmt = ‘update statistics [' + @tname + ']' + ', with full scan'
exec (@execstmt)
fetch c1 into @tname
end
deallocate c1
close c1
Siebel Connection Pooling
Object Manager configuration for 1,000 Call Center users
Concurrent Users
1,000 Call Center Users
Anonymous Users
150
MaxTasks
1,200
Object
Manager
Level
100:1
MaxMTServers
12
MinMTServers
12
Ratios sensitive to
average Think Times
10:1
MinTrxDBConns
MaxSharedDbConns
MinSharedDbConns
120
Siebel OM/DB Connection Pooling
 Siebel DB Connector creates multiple ODBC statements
handles per SQL Server connection
 If MaxSharedDbConns = 10 then 10 Siebel tasks share a
DB connection
 If MaxSharedDbConns = -1 then each task uses one DB
connection
10:1 Siebel OM
DB Connection Pooling
One Siebel task
can map to n
statements
statement
Thread/T
ask
connection
Process
Server
connection
process
Sales
Object
Manager
Siebel Server
SQL Server
SQL connection can map to a OS thread or fiber
Siebel OM/DB Connection Pooling disabled
connection
connection
connection
connection
connection
Thread
statement
connection
Process
connection
connection
connection
connection
connection
connection
Server
Sales
Object
Manager
Siebel Server
SQL Server
SQL connection can map to a OS thread or fiber
process
Data for Lowest Price/Performance
# Server
System
CPU's
OS
Database
tpmC
$/tpmC
Availability
Date
1
Pow erEdge 2650
Window s Server 2003 Server
SQL Server 2000 Std Ed.
20,109
$2.28
1/14/2004
2
HP ProLiant DL380G3-2P
Window s Server 2003 Ent Ed.
SQL Server 2000 Ent Ed. SP3
43,231
$3.71
5/27/2003
4
QuatreX-64 Server 4P
Window s Server 2003 Ent Server
SQL Server 2000 Ent Ed. SP3
82,226
$2.72 10/21/2003
4
rp server rx5670
HP UX 11.iv2 64-Bit Base OS
Oracle Database 10G Std Edition
131,640
$7.25 12/31/2003
8
IBM eServer xSeries 445 8P c/s
Window s Server 2003 Datacenter Ed.
SQL Server 2000 Ent Ed. SP3
139,154
$5.07 12/31/2003
8
IBM eServer pSeries 660 Model 6M1
IBM AIX 4.3.3
Oracle 9i Database Ent Ed. 9.0.1
105,025
$23.45
9/21/2001
16
Unisys ES7000 Orion 540 Ent Server
Window s Server 2003 Datacenter Ed.
SQL Server 2000 Ent Edition
181,280
$5.85
5/30/2003
16
Fujitsu PRIMEPOWER 850
Sun Solaris 8
Sybase Adaptive Server Ent v12.5
112,286
13.44
8/31/2002
32
Unisys ES7000 Orion 540 Ent Server
Window s Server 2003 Datacenter Ed.
SQL Server 2000 Ent Edition
252,920
$7.22
7/22/2003
32
IBM eServer pSeries 690 Turbo 7040-681 IBM AIX 5L V5.2
IBM DB2 UDB 8.1
763,898
8.31
11/8/2003
64
HP Integrity Superdome
Window s Server 2003 Datacenter Ed.64-bit SQL Server 2000 Ent Ed. 64-bit
786,646
$6.49 10/23/2003
64
HP Integrity Superdome
HP UX 11.iv2 64-Bit Base OS
824,165
8.28 12/31/2003
Source: www.tpc.org as of 27-August-03, complete details for TPC-C Position
Oracle Database 10G Ent Edition
Query Analyzer
Reading Graphical Execution Plan Output
Query Plan
Sequence of Steps
Member.corp_no
Cost 9%
SELECT
Cost: 0%
Bookmark Lookup
Cost: 8%
Hash Match Root…
Cost 28%
Index Seek
Scanning a particular range of rows from a
non-clustered index.
Physical operation:
Logical operation:
Row count:
Estimated row sizes:
I/O cost:
CPU cost:
Number of executes:
Cost:
Subtree cost:
Index Seek
Index Seek
414
24
0.00706
0.000605
1.0
0.007675(6%)
0.00767
Argument:
OBJECT: ([credit].[dbo].[member].[fname]),
SEEK: ([member],[firstname] >=‘Rb’ AND
[member],[firstname] <‘T’) ORDERED
Filter
Cost: 0%
Member.fname
Cost: 10%
Profiler

Monitors SQL Server events

KB: Q224587
Profiler Demo
Sysprocesses

Each user has an associated row in the system table
master..sysprocesses.

The stored procedure sp_who provides a list of these user
connections or threads as well as other connection information
such as command, resource, wait types, wait time and status.
When a thread waits, the columns waittype (binary(2)), waittime
(int) and lastwaittype (nchar(32)) and waitresource.. The values
for waittype and lastwaittype columns are set by memory
structures in SQL Server.

Lastwaittype is a character description of the last wait type for
this thread.

It is not reset until another wait state occurs. Thus, a non-blank lastwaittype
means the thread had at least one wait state.

The current wait status is recorded in the waittype column.

If the waittype is non-zero, the lastwaittype and waittype will be
equivalent and indicate the current waitstate for the SPID. If
waittype is 0x00, this means the thread is currently running.