* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Siebel Systems & Microsoft
Survey
Document related concepts
Oracle Database wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Tandem Computers wikipedia , lookup
Microsoft Access wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Clusterpoint wikipedia , lookup
Database model wikipedia , lookup
Team Foundation Server wikipedia , lookup
Relational model wikipedia , lookup
Transcript
SQL Server: Performance & Balanced System Design By Frank McBath [email protected] Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Dealing with Complexity Key Theme – “Simple but NOT simplistic” Static configuration parameters replaced with dynamic algorithms employing adaptive feedback Administrative control retained to manage system wide resources You can still constrain the amount of memory used by SQL Server – if you want to Dynamic Control Dynamic control algorithms maintain “near optimal” values in response to workload Control Value If you tune at this point in time you pick this value. Maybe you pick 2 values and reconfigure system for each “use” Time Instantaneous optimal value of control value Dynamic Control Analogy Ignition Timing: Vacuum Advance: user controlled by lever Classic “knob” Uses simple feedback Major advance Full Feedback: Continuously adjusted Monitors: temp, speed, engine response, etc. Memory, CPU & Clustering Limits Windows Server 2003 and SQL Server 2000 Windows Server 2003, Standard Edition Windows Server 2003, Enterprise Edition Windows Server 2003, Datacenter Edition 32-bit 32-bit 32-bit RAM (GB) 2 2 2 CPU 4 4 4 Max Resource Supported SQL Server 2000, Standard Edition 64-bit 64-bit 1 With Address Windowing Extensions (AWE) in SQL Server 2000 Cluster Nodes SQL Server 2000, Enterprise Edition 3 32 CPU 4 8 32 4 4 Cluster Nodes SQL Server 2000, 64-bit Edition 1 RAM (GB) RAM (GB) 64 1 64 512 CPU 8 64 Cluster Nodes 8 8 Scalability Optimized for Windows Server 2003 and Itanium Great performance Manageability T-SQL code-compatibility with SQL Server 2000 8 node clustering support Same on-disk format as 32-bit for easy migration One setup for database & OLAP based on Windows Installer technology Compelling alternative to expensive Unix solutions Cost Savings Large memory addressability (up to 32 TB) Nearly unlimited virtual memory (up to 8 TB) I/O savings due to larger memory buffer pools The highly scalable database platform for memory intensive, performance-critical business applications Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary PSPP Results as of Oct 26, 2003 10/08/03 - 5,000 Concurrent User on Unisys ES7000 server and Microsoft SQL Server 2000 10/07/03 - 32,000 Concurrent User on HP-UX servers 10/07/03 -12,000 Concurrent User on HP ProLiant and Integrity servers and Microsoft SQL Server 2000 (64-bit) 06/03/03 - 10,000 Concurrent User on HP-UX servers 04/24/03 - 30,000 Concurrent User on Unisys ES7000 / 2000 Series of servers and Microsoft SQL Server 2000 (64-bit) 02/05/03 - 5,000 Concurrent User on IBM eServer xSeries and Microsoft SQL Server 2000 10/21/02 - 4,500 Concurrent User on IBM eServer xSeries and IBM DB2 UDB 06/24/02 - 30,000 Concurrent User on IBM eServer pSeries and IBM DB2 UDB World-class performance 30,000 concurrent Users A typical PSPP environment: HP 12K 12,000 User Benchmark on HP/Windows/SQL64 – resource utilization Niode Functional Use Average CPU (%) Utilization Average Mem ory Utilization (MB) 4 x Proliant DL760 Web Server – Application Requests 8% 600 3 x Proliant BL20p Web Server – Application Requests 7% 500 1 x Proliant DL760 Web Server – HTTP Adapter, WF 9% 400 1 x Proliant 6400R Siebel Gatew ay Server 3% 200 4 x Proliant DL580 Siebel Application Server – End Users 13% 5,000 8 x Proliant BL40p Siebel Application Server – End Users 11% 4,700 1 x Proliant DL580 Siebel Application Server – EAI HTTP Adapter+ WF 25% 2,200 1 x Proliant DL760 Siebel Application Server – EAI-MSMQ Adapter 21% 916 1 x Proliant BL20p 1 x Integrity rx5670 Siebel Application Server – AM Microsoft SQL Server 2000 (64-bit) 3% 47% 80 13,300 12,000 User Benchmark on HP/Windows/SQL64 Concurrent Users Workload Avg Operation Business Num ber Response Tim e to Transactions of Users Load Runner (sec) Throughput / Hour Projected Transactions 8 Hour Day Sales / Service Call Center 8,400 0.137 43,662 349,300 eChannel (PRM) 1,200 0.131 16,130 129,037 eSales 1,200 0.144 8,164 65,313 eService 1,200 0.162 15,462 123,694 N/A 83,418 667,344 12,000 Totals Server Component Throughput Workload Assignm ent Manager Business Transactions Throughput / Hour Projected Transactions 8 Hour Day 62,012 496,096 EAI - HTTP Adapter 496,056 3,968,448 EAI - MQ Series Adapter Workflow Manager 294,539 116,944 2,356,312 935,552 SQL64 on a 4x 1.5 GHz Itanium2 HP Integrity used 47% CPU and 13.3 GB memory proving unprecedented price/performance for Siebel Oracle 10K vs SQL Server 12K on the DB-tier Oracle supported 10K users on rp8400 with 16x CPU 875Mhz with Oracle 9.x/Hp-ux posting 35% CPU and 18.2GB memory. MSSQL supported 12K users on rx5670 with 4x CPU 1.5Ghz with sql2K/windows2003 posting 47% CPU and 13.3GB memory. Result: 17.95% less CPU** 26% less memory 60% less cost 20% more users (12K vs 10K) SQL Server 2000 64-bit did more with less. *HP rx5670 is around $50K on HP web if you pay the full price. *HP rp8400 base price $124K ** rp8400 16 CPU SpecInt 98.2, rx5600 4 CPU SpecInt 60 ** (cont) (98.2 * 35%) = 34.37 , (60 * 47%) = 28.2 ** (cont) 1 – (28.2/34.37) = 17.95% ** www.spec.org ** Scott Hall Slide Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Tuning Tools Windows System Monitor – PERFMON.EXE System Monitor in the Performance console, started from Administrative Tools Query Analyzer – ISQLW.EXE Graphical showplan Statistics profile Profiler- PROFILER.EXE Spot problematic queries Use the Tuning or Duration templates Monitor the overhead carefully on your system Index Tuning Wizard – Particularly for EIM initial Exploring I/O with System Monitor Make sure you set DISKPERF –Y on command line to get counters dumped Performance Object: Physical Disk Counters: %Disk Time Number for entire logical drive should be less than 100% Avg Disk Queue Length (system requests on avg. waiting for disk access) Want 0 If it is above 2 (def. above 3), look into it See if it’s sustained queueing or temporary Avg. Disk Read/Write /sec (diff counters, and remember Logical vs. Physical) Nice to have: 5 – 7 ms (might be optimistic) Realistic (today’s technology): 20 – 25 ms on a moderately loaded system Log device service write times should be below 20 ms Technology dependent … See BOL (index “Bottlenecks” then “Monitoring Disk Activity”) for some more tips System Monitor Useful Counters Processor - % Processor Time Physical Disk - %Disk Time, Avg. Disk Queue Length Memory – Available MBytes System – Context Switches / sec SQL Server Locks – Lock Waits/sec, Number of Deadlocks/sec SQLServer: Access Methods Full Scans/sec, Page Splits/sec, Table Lock Escalation/sec SQLServer: Buffer Manager Buffer Cache Hit Ratio, Lazy Writes/sec, Page Reads/sec, Page Writes/sec, ReadAhead Pages/sec SQLServer: Databases - Transactions/sec SQLServer: General Statistics - User Connections Q150934 – How to Create a Performance Monitor Log Profiler Terminology Template Trace Limits the results (Equal, Not like, etc…) Event Category Captures data based upon selected events, data columns, and filters Filter Defines criteria for what to monitor Saved in .tdf file Defines the way events are grouped Event Action generated within SQL engine Profiler Use Built-in templates Find the worst-performing queries Filter by duration Identify the cause of a deadlock Monitor stored procedure performance Audit activity – C2 audits Reorder your output columns by Duration, CPU, read, writes, textdata, etc. Profiler in Production Can be very CPU intensive My experience in production: 8x at 100% Filter! Filter! Filter! Let the computer tell you what’s going foul To proactively find out what is going wrong Filter on Duration > 30,000ms Run for 24 hours This will show you all the poor running queries on your system. Can’t make it run faster? Look at IO’s in loops or running all the time. Repack to higher fill factor. 5 to 4 IO’s is a 20% increase. McBath’s Oil Argument Do you notice your car running better when you change your oil? No. But your car sure does. Make your database server run as efficient as possible. Makes it more scalable. More with less. Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Siebel Database Over 2,300 Tables Many with over 120 columns per table 2,200 Clustered Indexes (ROW_ID) 10,500 Non-Clustered Indexes Over Indexed, many NULL indexes 10,000+ Default Constraints 25,000+ Check Constraints Very few Stored Procedures Some Triggers used by workflow / assignment manager Siebel Logical Schema vs. Physical Schema Siebel database schema is designed as a cross platform logical schema, which is managed by Siebel Tools The logical schema is translated to a physical schema by using Siebel database utilities, such as DDLIMP The logical schema may be altered and mapped to a compatible physical data type depending on the DB platform and code page Don’t Be Afraid to Add/Change Indexes… You almost always have to. Work closely with Siebel Expert Services Make sure your Siebel meta data is in sync with SQL Server meta data… or bad things can happen next time you DDLSYNC… See examples in the next slides Reality (I) Out of Box… A Large Customer… sp_helpindex EIM_CONTACT3 sp_helpindex EIM_CONTACT3 EIM_CONTACT3_T01 EIM_CONTACT3_T01 EIM_CONTACT3_T02 EIM_CONTACT3_U1 EIM_CONTACT3_T03 EIM_CONTACT3_T04 EIM_CONTACT3_T05 EIM_CONTACT3_T06 EIM_CONTACT3_T07 EIM_CONTACT3_T08 EIM_CONTACT3_T09 EIM_CONTACT3_T10 EIM_CONTACT3_T11 EIM_CONTACT3_T12 EIM_CONTACT3_T13 EIM_CONTACT3_T14 EIM_CONTACT3_T15 EIM_CONTACT3_T16 EIM_CONTACT3_T17 EIM_CONTACT3_T18 EIM_CONTACT3_T19 EIM_CONTACT3_T20 EIM_CONTACT3_U1 Reality (II) S_CONTACT_EI S_CONTACT_F10 S_CONTACT_F11 S_CONTACT_F12 S_CONTACT_F13 S_CONTACT_F15 S_CONTACT_F2 S_CONTACT_F3 S_CONTACT_F4 S_CONTACT_F5 S_CONTACT_F6 S_CONTACT_F7 S_CONTACT_F8 S_CONTACT_II S_CONTACT_M1 S_CONTACT_M11 S_CONTACT_M12 S_CONTACT_M13 S_CONTACT_M14 S_CONTACT_M15 S_CONTACT_M16 S_CONTACT_M17 S_CONTACT_M18 S_CONTACT_M19 S_CONTACT_M2 S_CONTACT_M20 S_CONTACT_M21 S_CONTACT_M22 S_CONTACT_M3 S_CONTACT_M4 S_CONTACT_M6 S_CONTACT_M8 S_CONTACT_M9 S_CONTACT_P1 S_CONTACT_U1 S_CONTACT_U2 S_CONTACT_V1 S_CONTACT_V2 S_CONTACT_V3 S_CONTACT_V5 S_CONTACT_EI S_CONTACT_F6_X S_CONTACT_II S_CONTACT_M1 S_CONTACT_M50 S_CONTACT_M8 S_CONTACT_ML1_X S_CONTACT_ML2_X S_CONTACT_ML3_X S_CONTACT_ML4_X S_CONTACT_ML5_X S_CONTACT_ML6_X S_CONTACT_P1 S_CONTACT_PREM01_X S_CONTACT_PREM02_X S_CONTACT_U1 S_CONTACT_U2 S_CONTACT_V3 Indexes in RED were custom. Poor Indexes Get rid of 100% NULL indexes Cost a lot on INSERTS/UPDATES/DELETES, ex. EIM Cost disk space Cost tape space when you back them up Only time one might be used: on an aggregate. It’s cheaper than a full scan. Rare, though. Stored Procedure below will examine the indexes on the top 100 tables for indexes that probably will not be used. Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Siebel OM Query Execution and Fetching Mechanism 20 tables in a join… Siebel OM uses Server side API cursors For List applet functionality i.e. to maintain user state and support pending result sets To support multiple active statements per connections Fast Forward cursor with auto-fetch or Dynamic cursor when accessing text columns Sometimes there is an implicit conversions to Keyset (order by not covered by index ) Average fetch size is 3 or 4 rows – this is computed by dividing the ODBC buffer size by row size Siebel uses ODBC Prepare/Execute Example: Select * from table where x = ? What it looks like… SQL Fetch Mechanism sp_cursorprepe x (….., 3) Siebel OM 3 rows sp_cursorfetch (…,3) 3 rows c1 c2 c3 c4 c5 a b c d e f g h i j a b c d e f g h i j a b c d e f g h i j Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Typical Siebel OM Query declare @P1 int set @P1=-1 declare @P2 int set @P2=0 declare @P3 int set @P3=28688 Fast Forward, Parameterized, Auto Fetch, Auto Close (undocumented and subject to change) declare @P4 int set @P4=8193 declare @P5 int set @P5=10 exec sp_cursorprepexec @P1 output, @P2 output, N'',N' SELECT T1.LAST_UPD_BY, T1.ROW_ID, T18.PRTNR_TYPE, T13.CREATED_BY, T2.ASGN_USR_EXCLD_FLG, . . . T2.PAR_OU_ID FROM dbo.S_PARTY T1 INNER JOIN dbo.S_ORG_EXT T2 ON T1.ROW_ID = T2.PAR_ROW_ID INNER JOIN dbo.S_ACCNT_POSTN T3 ON T2.PR_POSTN_ID = T3.POSITION_ID AND T2.ROW_ID = T3.OU_EXT_ID INNER JOIN dbo.S_PARTY T4 ON T3.POSITION_ID = T4.ROW_ID LEFT OUTER JOIN dbo.S_ORG_EXT T5 ON T2.PAR_OU_ID = T5.PAR_ROW_ID LEFT OUTER JOIN dbo.S_PRI_LST T6 ON T2.CURR_PRI_LST_ID = T6.ROW_ID LEFT OUTER JOIN dbo.S_POSTN T7 ON T2.PR_MGR_POSTN_ID = T7.ROW_ID LEFT OUTER JOIN dbo.S_USER T8 ON T7.PR_EMP_ID = T8.ROW_ID LEFT OUTER JOIN dbo.S_ORG_EXT T9 ON T2.PAR_BU_ID = T9.PAR_ROW_ID LEFT OUTER JOIN dbo.S_ORG_EXT T10 ON T1.PAR_PARTY_ID = T10.PAR_ROW_ID LEFT OUTER JOIN dbo.S_ORG_PRTNR T11 ON T1.ROW_ID = T11.PAR_ROW_ID LEFT OUTER JOIN dbo.S_ORG_EXT_SS T12 ON T1.ROW_ID = T12.PAR_ROW_ID LEFT OUTER JOIN dbo.S_BU T13 ON T1.ROW_ID = T13.PAR_ROW_ID LEFT OUTER JOIN dbo.S_OU_PRTNR_TIER T14 ON T2.PR_PRTNR_TIER_ID = T14.ROW_ID LEFT OUTER JOIN dbo.S_ASGN_GRP T15 ON T2.PR_TERR_ID = T15.ROW_ID LEFT OUTER JOIN dbo.S_INDUST T16 ON T2.PR_INDUST_ID = T16.ROW_ID LEFT OUTER JOIN dbo.S_ADDR_ORG T17 ON T2.PR_ADDR_ID = T17.ROW_ID LEFT OUTER JOIN dbo.S_OU_PRTNR_TYPE T18 ON T2.PR_PRTNR_TYPE_ID = T18.ROW_ID LEFT OUTER JOIN dbo.S_POSTN T19 ON T3.POSITION_ID = T19.PAR_ROW_ID LEFT OUTER JOIN dbo.S_USER T20 ON T19.PR_EMP_ID = T20.PAR_ROW_ID LEFT OUTER JOIN dbo.S_ORG_SYN T21 ON T2.PR_SYN_ID = T21.ROW_ID LEFT OUTER JOIN dbo.S_ORG_BU T22 ON T2.BU_ID = T22.BU_ID AND T2.ROW_ID = T22.ORG_ID LEFT OUTER JOIN dbo.S_PARTY T23 ON T22.BU_ID = T23.ROW_ID LEFT OUTER JOIN dbo.S_ORG_EXT T24 ON T22.BU_ID = T24.PAR_ROW_ID WHERE ((T2.PRTNR_FLG != ''N'') AND ((T13.BU_FLG = ''N'' OR T13.BU_FLG IS NULL) AND T2.PRTNR_FLG = ''Y'')) OPTION(FAST 40)', @P3 output, @P4 output, @P5 output Build plan and return 40 rows ASAP Why do I get a different query plan in the Query Analyzer ? Bind value (Prepare Execute model) Hard Coding Values instead of binding at Run Time Cursor (SQL Server API cursor) Not putting the “ODBC Wrapper” SQL hint (Fast 40) Not including compiler options Text column Implicit Cursor Conversion Table Spools in one plan, but not the other Capture on Implicit Cursor Event in Profiler Also capture on “integer data” column (N)TEXT Columns (N)TEXT column may cause performance problems In the Siebel database schema A logical TEXT data type is always translated to a physical (N)TEXT column A VARCHAR data type can be translated to either a (N)VARCHAR column or a (N)TEXT column VARCHAR(2000+) is translated to a (N)TEXT column One size fits all: Implicit Cursor Conversions One type of cursor is requested, but it cannot be fulfilled in it’s native call. Rather than fail, SQL Server converts internally. Performance problems. For example, Siebel uses an “option fast 40” Fast forward, read only requested with an ORDER BY, yet no index on the WHERE clause that is ordered. SQL Server converts to a KEYSET cursor which spools off to TEMPDB for the sort. Fix: make an index that matches the ORDER BY. KEYSET conversion goes away. SQL Profiler: Event -> Cursors -> CursorsImplicitConversion Profiler Properties ODBC implicit cursor conversions: BOL Query Repro: Quick and Dirty Capture on RPC: Starting Event on Profiler Cut and paste it into Query Analyzer 99% of the time it will give you the same plan that is coming out of Siebel. DON’T: Spool Siebel out at the client, hard code the values and put into Query Analyzer. Probably won’t work For example, it won’t have the OPTION FAST 40 How to make the Query in QA print 'declaring variables' declare @P1 int declare @P5 int declare @P6 int set @P1=NULL -- set @P5=28688 -- SCROLLOPT 28676 = 16384 (AutoClose) + 8192 (AutoFetch) + 4096 (Parameterized) + 4 (Forward Only) set @P5=28676 set @P6=8193 print 'running sp_cursorprepare' exec sp_cursorprepare @P1 output , N'@P1 varchar(30)' -- , N'SELECT * FROM authors WHERE au_lname like @P1 OPTION (FAST 1)' , N'SELECT * FROM authors WHERE au_lname like @P1' ,1 , @P5 output , @P6 output print 'declaring more variables' declare @P2 int declare @P3 int declare @P4 int set @P3 = 1 set @P2=NULL set @P3=24592 -- SCROLLOPT - 24592 = 16384 (AutoClose) + 8192 (AutoFetch) + 4 (Forward Only) set @P4=8193 set @P5=15 print 'executing cursor' exec sp_cursorexecute @P1, @P2 output, @P3 output, @P4 output, @P5 output, 'R%' print 'select the results from the cusor execute' select @P1,@P2,@P3,@P4, @P5, @P6 Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Siebel and Update Statistics Problem: Siebel queries join a lot of tables A lot of the tables joined might be smaller than the threshold to execute an automatic update statistics Result: Bad plans for the join Stale statistics and EIM A Plan that tips over. 98 jobs run in 5 minutes. 2 run in 1 hour or… a lot longer. EIM running faster than Auto Update Stats can kick off. Solution run a manual update statistics on those tables Check rowmodctr in sysindexes for indid=1 See Appendix for code on auto update stats Example of Stale Statistics set statistics io on Table ‘S_CONTACT'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0. Table ‘S_OPPTY'. Scan count 4382, logical reads 21439, physical reads 650, read-ahead reads 877. Table ‘S_PARTY'. Scan count 2, logical reads 17573, physical reads 199, read-ahead reads 0. After UPDATE STATISTICS with sample of 10%: Table ‘S_CONTACT'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0. Table ‘S_OPPTY'. Scan count 192, logical reads 1440, physical reads 0, read-ahead reads 11. Table ‘S_PARTY'. Scan count 4, logical reads 1507, physical reads 3, read-ahead reads 59. Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Why Wait Types? Wait types will help define where your bottleneck is. They are seen in the master..sysprocesses table in a column called “waittype”. select waittype, * from master..sysprocesses There are all kinds of waittypes. For example, blocking due to database locks, network io, disk queueing, etc… The key to solving throughput is understanding what’s damming up the river. Wait Types If a thread goes into a sleep status, a wait state is set. The wait state is contained in master..sysprocesses in the columns waittype, and lastwaittype. Lastwaittype is a character description of the last wait state for this thread. It is not reset until another wait state occurs. Waittype is a varbinary wait state that is the current wait state. A wait time of 0 means the thread is currently running. See SQL_PERF.DOC for detailed information: Track_waitstats stored procedure Track_waitstats is a stored procedure that will capture waitstats from DBCC SQLPERF, and provide a ranking of descending order based on percentage. This is useful in identifying the greatest opportunites for performance improvements. See Appendix for Stored Procedure on Perf See the sample output below: Sample Output Commentary The above sample shows the majority share of wait time, 48%, being due to network IO waits. Improving network IO is the single largest opportunity for improving application performance. Other lesser opportunities in the above example include LCK_M_X (exlusive locks) and WRITELOG (transaction log). Exclusive lock waits account for almost 13% of total wait time. An examination of transaction management may offer clues as to whether improvements can be made here. WRITELOG means threads are waiting for physical writes to complete to the transaction log. Given the 11% writelog waits, a further analysis of PERFMON disk queues for the transaction log will confirm whether the IO capacity of the transaction log drives have trouble keeping up with write requests as shown by steady and high disk queues. Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Locating the Query Using the Most CPU Perfmon Counters: Threads: % Processor Time ID Process (NT PID) ID Thread (KPID) select spid from master..sysprocesses where kpid = <ID Thread> Select all SQLSERVR/0 to SQLSERVR/99 DBCC INPUTBUFFER(spid) Quick Estimate for Long Running Queries select datediff(mi, last_batch,getdate())'minutes',spid, waittype, cpu, physical_io, convert(char(15),hostname), convert(char(15),program_name), convert(char(20),getdate()),spid, last_batch, cmd from master..sysprocesses where spid > 50 and cmd not like '%WAIT%' and datediff(mi, last_batch,getdate()) > 1 order by last_batch Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary EIM & Data Loading I’m short changing you… I have a 2 hour presentation just on this subject! The following are just some “good ideas” on generic loading of data. Attached is a 50 ppt presentation on EIM and SQL Server Please contact me off line for information on this topic. Glad to work with you on it. Cursors, DTS, Loading Data and You Cursors are the ISAM / Dbase way of thinking about a problem. Think in set wise solutions Performance is drastically improved: 4:28 minutes to 23 seconds Show example with “set wise” and “cursor based” Compare Updateable vs. Fast Forward/Read Only Fair Warning: DTS DTS works great. Dev types tend to badly abuse it. “Let’s draw a cool picture…” Run it over a network Want to loop one row at a time ActiveX controls Rewrote a cleansing process from 15 minutes to 20 seconds by eliminating ActiveX See attached presentation on high perf data loading. Cursor DECLARE scrub_cursor CURSOR <FAST_FORWARD> FOR SELECT y FROM table_x <nolock> order by y -- 4:23 minutes -- normal: ------------------------------------------------------ 2003-08-06 09:48:26.560 ------------------------------------------------------ 2003-08-06 09:52:49.230 -- 3:50 minutes -- fast forward: ------------------------------------------------------ 2003-08-06 09:56:16.140 ------------------------------------------------------ 2003-08-06 10:00:05.780 Set Wise Just as much “work” for SQL Server to crunch 1 row as 10K rows… select getdate() go update table_x set ROW_ID = '0' go select getdate() go -- 26 seconds -----------------------------------------------------2003-08-06 09:44:14.263 (943093 row(s) affected) -----------------------------------------------------2003-08-06 09:44:40.030 Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Balanced System Design All systems have a bottleneck… But a balanced system tries to maximize all system components And pushes the bottleneck to the most “expensive” component CPU is piece to maximize Costs $5000 each Hard to add more CPUs without changing whole hardware architecture System bus not fast enough Other pieces more readily expandable I/O Subsystem (I) Disks becoming bigger and bigger in terms of volume Access speed does not increase at the same rate Problem: Disk access speed essential for I/O performance Don’t maximize on volume but on # of spindles Typical disk performance I/O sec 8K 64K Sequential I/O 150 1.2 MB/sec 9.4 MB/sec Random I/O 100 0.8 MB /sec 6.3 MB/sec I/O Subsystem (II) Recommendation since ever: Log File separated from Data Files on RAID1 or RAID1+0 Data Files minimum protection RAID5 RAID5 has severe disadvantages in writing blocks due to the fact that checksum for block has to be read, calculated and written down Especially for applications with time sensitive import workload (like nightly EIM Extracts) RAID1+0 for data files are recommended I/O Subsystem (III) SAN storage backend Usually huge caches Own methods of building volumes to optimize data distribution over a high number of disks Issues: Caches do not speed up all kind of operations. Typical example: Backup or checkdb Highly complex data distribution on this hardware might hide the fact that portions of Data files and Log files end up on the same spindle Advertisement might indicate that a physical separation of Data files and Log files are not necessary anymore Still insist on physical separation of Date file and Log File I/O Subsystem (IV) Fiber-Channel and SCSI adapter have bandwidth limitations as well. Some are >100MB/sec according to spec These are max. values with burst and other special cases. In normal operations bottleneck appear way earlier PCI-Bus has limitations as well E.g. Profusion 8way architecture. Not every PCI slot has full throughput I/O Subsystem (V) How to detect I/O bottlenecks Avg. Disk Queue/sec with Perfmonitor Avg. sec/transfer with Perfmonitor Avg. sec/transfer should not be beyond 20ms for the data partitions and less than 10ms for transaction log partitions Attention: Due to SQL Server’s architecture the maximum queue for a dedicated transaction log partition (recommended) will not be higher than 3. Therefore Avg. Disk Queue/sec is not very helpful for analyzing performance issue on the transaction log partition Log stalls hold up transaction commit and can lead to significant concurrency issues I/O subsystem (VI) Select * from ::fn_virtualfilestats(-1,-1) Show stall time per I/O request on a per file basis Select DbId, FileId, IoStallMS/(NumberReads+NumberWrites) ‘ms/io’ from ::fn_virtualfilestats(-1,-1) Limits for data files the same. Due to different measurement value for log can be a little higher than with perfmon What the numbers say… Stall time per I/O This should gives a good idea on I/O Disadvantage of this solution is that the counters are not resetable. What happens then over time is that the immediate impact of some performance issues disapears in the clutter A more in depth look… SQL Server Profiler Counters: Physical disk avg disk sec/read Physical disk avg disk sec/write Takes into account disk queues If not queuing then you’d normally expect to see reads taking 4-10ms as low as 1ms (w/cache). If you know (in the absence of queuing) a read should take 6ms and you are seeing an average of say 15ms, then you have can see the effect of sustained queuing. Thus you have an IO bottleneck. Memory (I) With Windows 2000 Advanced Server up to 8GB real memory can be addressed (/PAE entry in boot.ini) Still one single process is limited on max 3GB But all processes can share the 8GB real memory SQL Server 2000 Enterprise Edition can leverage up to 8GB as well using AWE Memory (II) Does it make sense to use Datacenter Edition to leverage more than 16GB memory for SQL Server 2000 Typically not under a typical workload Only SQL Server data pages can reside outside the virtual address space of 3GB Lock structures, query plans, buffer headers and other highly frequented structures can not use AWE memory Recommendation: Usage of AWE up to 8GB by SQL Server is fine and can show great performance improvement Having /PAE enabled the system is not able to read or write 64K Tape I/O blocks Backups performed on a system without /PAE enabled are not readable on a system with /PAE Online backup against tape could take a little longer File Management Everlasting question: How many data files should be used? Ideally between 1 and 3 per disk array No need to restrict file size for performance reasons Minimum # of data files around also dependent on # CPUs Ratio of .25 - .5 times # of CPUs is sufficient to guarantee good performance Grow files manually to guarantee optimal proportional fill Sample SQL Disk Configuration SQL Server Fibre Controller Fibre Controller Fibre Switch Fibre Switch SCSI SCSI SCSI LTO 12 100gb logical disks (Raid 0+1) 1 SQL file per disk (H: S:) 3 Drive LTO Array StorageWorks disk subsystem LTO 1.4 TB Backed up in > 3 hours (130 MB/Sec) Faster than backing up to Disk Online Backup No user performance impact Network Performance Load all the data locally Named Pipes vs TCP/IP Named Pipes if on the same server will run faster Can control through the SQL Client Network utility Agenda Introduction Benchmarking Tools & Monitoring Siebel Database Siebel Queries Siebel Query Repro Siebel & Statistics Wait Types Locating Long Running Queries Optimizing Data Loading Balanced System Design - Network Performance - IO - Memory - File Management Summary Summary of Tuning SQL 2000 Tuning Specifics Disable all unnecessary Siebel component groups to reduce system resources Data file placement – Data, Transaction Log, Tempdb Keep on separate spindles Pre-allocate space for DB and Tempdb Autogrow on Physically isolate SQL Transaction Log on mirrored drive with no other I/O Raid: use many spindles to allow parallel retrieval of data Use Siebel database connection pooling to reduce the memory footprint on the databas server Indexing Clustered index scans vs. Clustered index seeks Use /3GB in windows boot.ini on SQL32 for >5K Siebel users Common Mistakes… Not believing in Dynamic Control Believing too much in Dynamic Control Not paying attention to indexing Not paying attention to resources Lack of understanding Index hints Not utilizing autostats Recovery mechanisms Fragmentation Inappropriate use of Cursors during data loading – non Siebel processes SQL 2000 Tuning Specifics Maintain accurate statistics AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS ON Manually Update stats after large data modifications DBCC SHOW_STATISTICS Isolate specific tables / indexes into SQL Filegroups Siebel lists candidate tables in documentation (eg. S_DOCK_TXN_LOG) Separate EIM interface tables from Base tables Allow dynamic memory growth Index fragmentation DBCC SHOWCONTIG DBCC INDEXDEFRAG – online operation Tune SQL Query plan analysis Cursor implicit conversions SQL Server tuning guidelines Index fragmentation DBCC SHOWCONTIG DBCC INDEXDEFRAG – online operation DBCC DBREINDEX – offline operation (ex. can block) Windows System Monitor System Monitor in the Performance console, started from Administrative Tools Profiler Spot problematic queries Use the Tuning or Duration templates Monitor the overhead carefully on your system Graphical showplan in Query Analyzer Index Tuning Wizard – Particularly for EIM initial SQL Server tuning guidelines Data file placement – Data, Transaction Log, Tempdb Keep on separate spindles Pre-allocate space Autogrow off or very small increment on Transaction Log and Tempdb Physically isolate SQL Transaction Log on mirrored drive with no other I/O Indexing too many too few Clustered index scans vs. Clustered index seeks Maintain accurate statistics AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS ON Manually Update stats after large data modifications Isolate specific tables / indexes into SQL Filegroups Siebel lists candidate tables in documentation (eg. S_DOCK_TXN_LOG) Separate EIM interface tables from Base tables Allow dynamic memory growth SQL Server tuning – Actual EIM case Problem: EIM initial processing 30,000 – 40,000 rows per hour Final Results: EIM initial processing 1,300,000 rows per hour Process: Standard Siebel Expert Services methodology Tools: Siebel SQL Tracing for EIM to show 10 most expensive queries Window System Monitor to measure I/O, CPU and memory SQL Server Profiler to capture database events Steps: Spot problematic queries – long running insert Memory, SQL settings all fine, I/O was poor due to HW problems Use Query Analyzer (Show Execution Plan) to determine whether additional index will help – in this case: No Minimize I/O by dropping indexes on Base Tables (S_CONTACT) – 400,000+ Separate data into batches by key ranges to avoid deadlocks and run 4 EIM processes in parallel – 1.3 Million rows / hour Recreate indexes, update stats, defrag table and indexes Investigate I/O subsystem Hardware / SAN Thanks! Thank you for attending! Frank McBath [email protected] Appendix Matter Performance Isolation and Resolution Poor Database Response Time Disk Subsystem Architecture: Controllers, Logs, Data Files, TempDB Spread Out The Load Type of Problem Watch Perfmon High CPU Utilization Query Perfmon: THREAD Counters High IO STATISTICS IO Look at Contig Space DBCC SHOWCONTIG Defrag: Space, Fill Factor DBCC REINDEX/ INDEXDEFRAG/ SELECT INTO Unpredictable Plan Look At Plan SHOWPLAN With RECOMPILE Book Mark Lookup STATISTICS PROFILE Look At Statistics DBCC SHOW_STATISTI CS Rearrange the Columns DBCC SHOW_STATISTI CS Hard Code Hint (INDEX = ) WITH RECOMPILE Covered Index UPDATE STATS with HigherSampling Frequency Create New Index Using Correct Index Selective Enough? “BOE” method Kill Run Away Job/ SPID Scope Query Results Down: Tighten up WHERE Supported Platforms: OS/DB Current Siebel 7.5 Platform Support Database Server Platforms Microsoft SQL Server 2000 Microsoft SQL Server 2000 (64-bit) eBusiness Server Platforms Microsoft Windows NT 4.0 Microsoft Windows 2000 Server / Advanced Server Microsoft Windows 2000 Datacenter Cluster Software Microsoft Clustering Service for Windows NT 4.0 Microsoft Clustering Service for Windows 2000 Web Servers Microsoft IIS 4.0 Microsoft IIS 5.0 HI Mode Applications Browsers: Microsoft IE 5.5, Microsoft IE 6.0 Operating Systems: Windows NT / 2000 / XP SI Mode Applications Browsers: Microsoft IE 5.01 / 5.5 / 6.0, MSN 8.0 Operating Systems: Windows 95 / 98 / ME / NT 4.0 / 2000 / XP New platforms or versions are highlighted in RED Planned Siebel 7.7 Platform Support – Q1, 2004 Database Server Platforms Microsoft SQL Server 2000 Microsoft SQL Server 2000 (64-bit) eBusiness Server Platforms Microsoft Windows 2000 Server / Advanced Server / Datacenter Microsoft Windows 2003 Server / Advanced Server / Datacenter Cluster Software Microsoft Clustering Service for Windows 2000 Microsoft Clustering Service for Windows 2003 Web Servers Microsoft IIS 5.0 Microsoft IIS 6.0 HI Mode Applications Browsers: Microsoft IE 5.5, Microsoft IE 6.0 Operating Systems: Windows NT / 2000 / XP SI Mode Applications Browsers: Microsoft IE 5.01 / 5.5 / 6.0, MSN 8.0 Operating Systems: Windows 98 / ME / NT 4.0 / 2000 / XP New platforms or versions are highlighted in RED Performance & Scalability Improved Performance Improved Scalability Up to 64-way SMP Up to 512 GB RAM NUMA Optimization Intel Hyper-Threading Technology Native 64-bit support (Itanium) Hardware vendor support • 64-way HP SuperDome Support for New Scalable Hardware • 32-way NEC Express5800 • 32-way Unisys ES7000 Orion 130 • 16-way IBM eServer xSeries 440 Additional resources Siebel Supportweb http://ebusiness.siebel.com/supportweb Periodically check for new Tech Alerts and Updates SQL Server Books Online KB Articles: 298475 : Information Required to Successfully Troubleshoot Application Performance Issues 243589 : HOW TO: Troubleshoot Slow-Running Queries on SQL Server 7.0 or Later SQL Server User groups Books: SQL Server 2000 Performance Tuning, Microsoft Press • Good sections on RAID, Capacity Planning, System Tuning Inside SQL Server 2000, Microsoft Press • The one book to have on SQL Server Creating server based trace DECLARE @RC int, @TraceID int, @on BIT EXEC @rc = sp_trace_create @TraceID output, 0, N'C:\test7' SELECT RC = @RC, TraceID = @TraceID SELECT @on = 1 exec sp_trace_setevent @TraceID, 10, 1, @on exec sp_trace_setevent @TraceID, 10, 2, @on exec sp_trace_setevent @TraceID, 10, 3, @on exec sp_trace_setevent @TraceID, 10, 6, @on exec sp_trace_setevent @TraceID, 10, 7, @on exec sp_trace_setevent @TraceID, 10, 8, @on exec sp_trace_setevent @TraceID, 10, 9, @on exec sp_trace_setevent @TraceID, 10, 10, @on -- Set the Filter EXEC sp_trace_setfilter 1, 10, 0, 6, N'MS%‘ -- Start Trace (status 1 = start) exec @RC = sp_trace_setstatus @TraceID, 1 GO Additional resources Siebel Supportweb http://ebusiness.siebel.com/supportweb Periodically check for new Tech Alerts and Updates KB Articles: 298475 : Information Required to Successfully Troubleshoot Application Performance Issues 243589 : HOW TO: Troubleshoot Slow-Running Queries on SQL Server 7.0 or Later SQL Server User groups Books: SQL Server Books Online – look here first SQL Server High Availability, Microsoft Press SQL Server 2000 Performance Tuning, Microsoft Press • Good sections on RAID, Capacity Planning, System Tuning Inside SQL Server 2000, Microsoft Press • The one book to have on SQL Server Siebel 7.5 - MCI Proof of Concept / Implementation Test demonstrated scalability of SQL Server 2000 Large anticipated customer workload 500,000 accounts 8,000 Concurrent Users Sub-second response times for majority of transactions Real world analysis Tested Siebel 7.5 implementation of Sales Force Automation Provided MCI with validation of proposed architecture for their Sales Force Transformation (SFT) Program Real world analysis using MCI’s hardware configuration and existing Siebel data Initial Scalability testing at the Accenture High Performance Scalability Lab in Cincinnati, OH From MCI’s John Heveran’s presentation at Siebel user week 2003 @ San Diego Siebel customers realize significant On-Going TCO Benefits Three year total cost of ownership comparison on Siebel deployments Resource SQL Server 2000 Competing Product Savings % Difference % Savings Hardware $47,602 $109,616 $62,014 57% 8% Software $103,596 $160,000 $55,404 35% 7% Hardware Maintenance $7,777 $21,652 $13,875 64% 2% Software Maintenance $58,161 $79,049 $20,888 26% 3% $791,704 $1,369,040 $577,336 42% 76% $2,279 $31,530 $29,251 93% 4% $1,011,119 $1,770,887 $758,768 43% 100% Ongoing Operations Support Activities Training Total Cost of Ownership http://www.microsoft.com/sql/evaluation/compare/migratingSiebelCRM.asp NerveWire study reveals a three-year ROI of 248% as a result of migrating Siebel CRM applications to SQL Server2000 on Windows 2000 from a competitive RDBMS on Unix. A company with sales of $2B/Yr. and a gross profit margin of 5% would need to increase sales by $20MM over 3 years to accomplish the same results! Siebel Scalability On Available Platforms Workload (User Type) AIX & DB2 Sales / Service Call Center Avg Operation Response Tim e to Load Runner (sec) Num ber of Users W2K & SQL2K HP-UX AIX & DB2 W2K & SQL2K HP-UX Business Transactions Throughput / Hour AIX & DB2 W2K & SQL2K HP-UX 20,000 20,000 22,400 0.148 0.295 0.116 122,041 121,425 116,571 eChannel (PRM) 4,000 4,000 3,200 0.182 0.185 0.212 27,615 27,619 42,890 eSales 3,000 3,000 3,200 0.233 0.207 0.242 17,134 17,157 21,703 eService 3,000 3,000 3,200 0.196 0.147 0.228 40,455 40,521 41,148 30,000 30,000 32,000 207,245 206,722 222,312 Totals Workload (Background Processing) Business Transactions Throughput / Hour AIX & DB2 Assignm ent Manager W2K & SQL2K HP-UX 38,599 37,693 22,817 EAI - HTTP Adapter 746,676 854,557 770,905 EAI - MQ Series Adapter Workflow Manager 545,472 96,299 728,745 97,585 540,845 - - - Metric DB Grow th (Proj GB/Month) AIX & DB2 W2K & SQL2K 250.00 290.00 DB Mem ory Used (GB) 18.10 25.40 31.10 Database Connections Web Server kbps per user 1,818 3,302 1,661 6.50 0.54 4.50 60,244 Note: 30,000 user tests are based on Siebel 7.0.3 and 32,000 test is based on 7.5.2; transaction mix is different between Siebel 7,0.3 and 7.5.2 test suites. N/A Resource Utilization by 30,000 and 32,000 Concurrent Users Test Node 30,000 Users CPU 84% 0.74 GB 11% 0.59 GB 70% 14.82 GB Siebel Servers (AM / EAI / WF) 2 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM 85% 0.54 GB 46% 1.054 GB 5 x HP rp5470 /w 4 875 MHZ & 16GB RAM 37% 0.816 GB 70% 0.070 GB 4 x HP rp8400 /w 16 x 875MHz & 64 GB RAM 82% 17.55 GB 1 x HP Superdome /w 32 x 875MHz & 128GB RAM 81% 34.70 GB 82% 2.90 GB 62% 31.10 GB Web Servers (EAI HTTP Requests) Siebel Servers (Object Managers) Siebel Servers (AM / EAI / WF) Database Server - IBM DB2 v7.2 1 x IBM p690 /w 32xRS64-IV 1.3GHz & 128GB RAM 5 x HP rp5470 /w 4 750 MHZ & 16GB RAM 1 x HP rp2470 /w 2 750 MHZ & 8GB RAM Siebel Servers (Object Managers) 5 x IBM p680 /w 24xRS64-IV 600MHz & 64GB RAM Mem Web Servers (User Requests) Web Servers (EAI HTTP Requests) 1 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM 32,000 Users CPU Mem Web Servers (User Requests) 3 x IBM p660 6H1 /w 6xRS64-IV 668MHz & 16GB RAM Node 23% 18.10 GB 1 x HP rp8400 /w 16 x 875MHz & 64GB RAM Database Server - Oracle 9.2.0.2 1 x HP Superdome /w 16 x 875MHz & 64GB RAM Node 30,000 Users CPU Mem 56% 0.184 GB 39% 0.035 GB 48% 3.132 GB 19% 1.805 GB 2 x Unisys ES2081 /w 8 x PIII 700MHz (EAI HTTP/MQ Series) 57% 0.810 GB Web Servers (User Requests) 8 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM Web Servers (EAI HTTP Requests) 1 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM Siebel Servers (Object Managers) 35 x Unisys ES2041 /w 4 x PIII 700MHz & 4GB RAM Siebel Servers (AM / EAI / WF) 2 x Unisys ES2041 /w 4 x PIII 700MHz (AM / WF) Database Server - MS SQL Server 2000 64-bit 1 x Unisys ES7000 Orion 130 /w 16 x Itanium 2 1GHz & 6467% GB RAM 25.74 GB Server consolidation / Mainframe class reliability Unisys ES7000 Siebel “In-a-Box” Typical Large Implementation Web Servers Application Servers Web Servers Siebel 7 Application Server and SQL Database Server Database Server Routers Routers Routers Unisys ES7000 Unisys “In-a-Box” Siebel 5,000 user PSPP Benchmark new! Monitoring Stored Procedure create proc track_waitstats (@num_times int=5,@delaymin int=1) as -- @num_times is the number of times to capture waitstats, default is 5 times -- default delay interval is 1 minute -- create waitstats table if it doesn't exist, otherwise truncate set nocount on if not exists (select 1 from sysobjects where name = 'waitstats') create table waitstats ([Wait Type] varchar(80), Requests numeric(20,1), [Wait Time] numeric (20,1), [Signal Wait Time] numeric(20,1), now datetime default getdate()) else dbcc sqlperf (waitstats,clear) -- clear out waitstats declare @i int,@delay varchar(8),@now datetime, @totalwait numeric(20,1),@endtime datetime,@begintime datetime select @i = 1 select @delay='00:' + right('0'+convert(varchar(2),@delaymin),2) + ':00' while (@i <= @num_times) begin truncate table waitstats insert into waitstats ([Wait Type], Requests, [Wait Time],[Signal Wait Time]) exec ('dbcc sqlperf(waitstats)') select @i = @i + 1 waitfor delay @delay end select @now=max(now),@begintime=min(now),@endtime=max(now), @totalwait=max([wait time]) from waitstats where [wait type] = 'Total' --- subtract waitfor, sleep, and resource_queue from Total select @totalwait=@totalwait - sum([wait time]) from waitstats where [wait type] in ('Waitfor','Sleep','Resource_Queue') and now = @now -- insert adjusted totals, rank by percentage descending insert into waitstats select '***total***',0,@totalwait,@totalwait,@now select 'start, end, duration'='start: ' + convert(varchar(20),@begintime,20) + ' end: ' + convert(varchar(20),@endtime,20) + ' duration (minutes): ' + convert(varchar(10), datediff(mi,@begintime,@endtime)) select [wait type],[wait time],percentage=cast (100*[wait time]/@totalwait as numeric(20,1)) from waitstats where [wait type] not in ('waitfor','sleep','resource_queue','total') and now = @now order by percentage desc Siebel and Update Statistics Code declare @tname sysname, @execstmt varchar(128) set nocount on declare c1 cursor for select object_name(id) from sysindexes where indid=1 and id> 100 and rows<500 and rowmodctr >0 open c1 fetch c1 into @tname while @@fetch_status<>-1 begin set @execstmt = ‘update statistics [' + @tname + ']' + ', with full scan' exec (@execstmt) fetch c1 into @tname end deallocate c1 close c1 Siebel Connection Pooling Object Manager configuration for 1,000 Call Center users Concurrent Users 1,000 Call Center Users Anonymous Users 150 MaxTasks 1,200 Object Manager Level 100:1 MaxMTServers 12 MinMTServers 12 Ratios sensitive to average Think Times 10:1 MinTrxDBConns MaxSharedDbConns MinSharedDbConns 120 Siebel OM/DB Connection Pooling Siebel DB Connector creates multiple ODBC statements handles per SQL Server connection If MaxSharedDbConns = 10 then 10 Siebel tasks share a DB connection If MaxSharedDbConns = -1 then each task uses one DB connection 10:1 Siebel OM DB Connection Pooling One Siebel task can map to n statements statement Thread/T ask connection Process Server connection process Sales Object Manager Siebel Server SQL Server SQL connection can map to a OS thread or fiber Siebel OM/DB Connection Pooling disabled connection connection connection connection connection Thread statement connection Process connection connection connection connection connection connection Server Sales Object Manager Siebel Server SQL Server SQL connection can map to a OS thread or fiber process Data for Lowest Price/Performance # Server System CPU's OS Database tpmC $/tpmC Availability Date 1 Pow erEdge 2650 Window s Server 2003 Server SQL Server 2000 Std Ed. 20,109 $2.28 1/14/2004 2 HP ProLiant DL380G3-2P Window s Server 2003 Ent Ed. SQL Server 2000 Ent Ed. SP3 43,231 $3.71 5/27/2003 4 QuatreX-64 Server 4P Window s Server 2003 Ent Server SQL Server 2000 Ent Ed. SP3 82,226 $2.72 10/21/2003 4 rp server rx5670 HP UX 11.iv2 64-Bit Base OS Oracle Database 10G Std Edition 131,640 $7.25 12/31/2003 8 IBM eServer xSeries 445 8P c/s Window s Server 2003 Datacenter Ed. SQL Server 2000 Ent Ed. SP3 139,154 $5.07 12/31/2003 8 IBM eServer pSeries 660 Model 6M1 IBM AIX 4.3.3 Oracle 9i Database Ent Ed. 9.0.1 105,025 $23.45 9/21/2001 16 Unisys ES7000 Orion 540 Ent Server Window s Server 2003 Datacenter Ed. SQL Server 2000 Ent Edition 181,280 $5.85 5/30/2003 16 Fujitsu PRIMEPOWER 850 Sun Solaris 8 Sybase Adaptive Server Ent v12.5 112,286 13.44 8/31/2002 32 Unisys ES7000 Orion 540 Ent Server Window s Server 2003 Datacenter Ed. SQL Server 2000 Ent Edition 252,920 $7.22 7/22/2003 32 IBM eServer pSeries 690 Turbo 7040-681 IBM AIX 5L V5.2 IBM DB2 UDB 8.1 763,898 8.31 11/8/2003 64 HP Integrity Superdome Window s Server 2003 Datacenter Ed.64-bit SQL Server 2000 Ent Ed. 64-bit 786,646 $6.49 10/23/2003 64 HP Integrity Superdome HP UX 11.iv2 64-Bit Base OS 824,165 8.28 12/31/2003 Source: www.tpc.org as of 27-August-03, complete details for TPC-C Position Oracle Database 10G Ent Edition Query Analyzer Reading Graphical Execution Plan Output Query Plan Sequence of Steps Member.corp_no Cost 9% SELECT Cost: 0% Bookmark Lookup Cost: 8% Hash Match Root… Cost 28% Index Seek Scanning a particular range of rows from a non-clustered index. Physical operation: Logical operation: Row count: Estimated row sizes: I/O cost: CPU cost: Number of executes: Cost: Subtree cost: Index Seek Index Seek 414 24 0.00706 0.000605 1.0 0.007675(6%) 0.00767 Argument: OBJECT: ([credit].[dbo].[member].[fname]), SEEK: ([member],[firstname] >=‘Rb’ AND [member],[firstname] <‘T’) ORDERED Filter Cost: 0% Member.fname Cost: 10% Profiler Monitors SQL Server events KB: Q224587 Profiler Demo Sysprocesses Each user has an associated row in the system table master..sysprocesses. The stored procedure sp_who provides a list of these user connections or threads as well as other connection information such as command, resource, wait types, wait time and status. When a thread waits, the columns waittype (binary(2)), waittime (int) and lastwaittype (nchar(32)) and waitresource.. The values for waittype and lastwaittype columns are set by memory structures in SQL Server. Lastwaittype is a character description of the last wait type for this thread. It is not reset until another wait state occurs. Thus, a non-blank lastwaittype means the thread had at least one wait state. The current wait status is recorded in the waittype column. If the waittype is non-zero, the lastwaittype and waittype will be equivalent and indicate the current waitstate for the SPID. If waittype is 0x00, this means the thread is currently running.