Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Siebel 7 Performance and Scalability Inside the Siebel Server Richard Sands Siebel Expert Services ©Siebel Systems 2003 – Do not distribute or re-use without permission Siebel 7 Performance and Scalability Inside the Siebel Server Introduction to Siebel Architecture How components scale Within servers Across servers Siebel Performance Hints Monitoring Diagnosis Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Siebel 7 Infrastructure Overview Wireless Web Browser User Interface Connected Web User (Employee) Connected Web User (External) Browser User Interface Browser User Interface PDA Mobile (Disconnected) Web User Browser User Interface Object Manager WAP Gateway Server Data Manager Web Server(s) Siebel Web Server Extension SIEB SYNC Local DB Gateway Name Server Central Load Dispatch Balancer Siebel Remote External Applications Siebel Enterprise Server Voice Interaction Siebel eAI Siebel Replication Object Manager Data Manager Object Manager Data Manager Regional Siebel DB Server Central Siebel DB Server Email Interaction Major Client Types All accessed through a browser High Interactivity (Employee facing) Very demanding on browser Can only run on strictly defined browser configurations Rich user interface Standard Interactivity (Customer facing) Less demanding on browser Can run on wide variety of browsers Standard web user interface Mobile Client Has local copy of Siebel database User-specific subset of data Local server functionality Quasi-web server Business Object and Data Manager functionality Uses High Interactivity interface Siebel Enterprise Server – SWSE Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Enterprise Server Siebel Server Component Siebel Server Component Siebel Web Server Extensions Web Server Plug-In Manages communications to Siebel Enterprise Includes cache for static files (images, etc) Siebel Enterprise Server – Gateway Server Serves as a single entry point Includes Siebel Gateway Name Server Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Siebel Enterprise configuration data (static) Siebel Enterprise operations data (dynamic) Optionally includes Resonate Central Dispatch Load balancing Enterprise Server Siebel Server Component Siebel Server Component Siebel Enterprise Server – Gateway Name Server Holds Enterprise Configuration Stores component definitions, parameters, and connectivity information Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Enterprise Server Siebel Server Component Siebel Server Component Stored in siebns.dat file Provides Connectivity information when Resonate not used Dynamically registers Siebel Server and component availability Siebel Enterprise Server – Resonate Central Dispatch Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Enterprise Server Siebel Server Component Siebel Server Component Load Balances Object Manager Components Provides resilience for load balanced components Provides session management Simplifies firewall configuration Not always needed Architecture Overview – Siebel Server Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Enterprise Server Siebel Server Component Siebel Server Component Framework for running server components Obtains configuration information from the Gateway Name Server Runs as a Windows service Siebel Enterprise Server is a logical grouping of Siebel Servers Architecture Overview – Server Components Web Server SWSE Gateway Server Gateway Name Server Resonate Central Dispatch Enterprise Server Siebel Server Siebel Server Component Component A type of program, executed as a task Examples: Object Manager Handling client requests Workflow Process Manager – Processing Workflows File System Manager Handling access to Siebel File System Architecture Overview – Server Component Types Different types of component run in different ways: Background Background mode components execute tasks to perform background operations for the Siebel Server. After a background mode component task starts, it runs until you explicitly stop the task, or until the Siebel Server itself is shut down. Interactive Interactive mode components start tasks automatically in response to client requests. Interactive mode component tasks execute for as long as the client maintains the session, and end when the client disconnects. Batch Batch mode components execute tasks in response to requests. Batch mode component tasks execute until they finish processing. Internal batch tasks are initiated by other Siebel Server tasks External batch tasks are initiated by Server Manager Architecture Overview – Component Execution Platforms Different types of component execute in different platforms: Single Threaded Single threaded components have one execution stream per process. So each operating system process supports a single Siebel Task. i.e. EIM Multi-threaded Multi-threaded components have multiple execution streams within a single process.. So each operating system process can support multiple Siebel Tasks. i.e. Object Managers Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Component Scalability How Components Scale in a Siebel 7 Enterprise Scaling within a server Multi-threaded components Single-threaded components Scaling across servers Load balancing Resonate central dispatch Load sharing Server Request Broker Server Request Processor Scaling Within a Siebel Server Single Threaded Components Create multiple processes (tasks) Some components are limited i.e. Transaction Processor – max 1 per server Workflow Monitor Agent – max 1 per policy group per Enterprise Can be started manually, through Server Request Broker, or automatically (‘Default Tasks’ parameter) Multi-Threaded Components Create multiple threads &/or processes (tasks) Control distribution through component parameters Multi-Threaded Components Can have multiple processes as well as multiple threads Important to control ratio of threads to processes Can have major impact on performance Determined primarily by rate of switches between threads 100:1 good starting point for Client Object Managers Assumes 30sec think time For 15 sec think time use 50:1 Can set additional processes to spawn on demand Will always start minimum number specified Will start additional processes as needed to maintain process:thread ratio Limit on maximum number of processes Memory Scalability Multi-Process, Multi-Threaded model All threads in a process share the same memory space Each Process has a separate memory space No single process needs a large memory space Operating system manages memory allocation If a single process needs more than 1GB there’s normally something wrong No need for large process memory model (‘/3GB’ switch) Can use Microsoft PAE for access to large memory capacities Operating system can allocate memory to each process Multi-Threaded Component Parameters Typically set per component Maximum number of tasks (MaxTasks) Maximum number of Tasks per component per server One thread per task Some additional background “system” threads - not counted by MaxTasks Maximum number of Multi-Threaded servers (MaxMTServers) An MTServer is a multi-threaded component process This defines the maximum number of MTServers per component per server Minimum number of Multi-Threaded servers (MinMTServers) This defines the minimum number of MTServers per component per server Sets the number of MTServers started on server startup Configuring the Object Managers Set MaxTasks = peak (concurrent users + anonymous users) Anonymous users are used for login screens before user authenticates Typically set to 10%-15% of concurrent user count Set MaxMTServers = MaxTasks / 100 An MTServer is equivalent to single instance of Object Manager 100 : 1 ratio is assuming “average” 30 second think time between user operations If average user think time is 15 seconds then ratio is 50 : 1 ( 50% of 100:1) If average user think time is 60 seconds then ratio is 200 : 1 (200% of 100:1) Set MinMTServers = MaxMTServers Setting MinMT Servers < MaxMTServers may cause delay of service for “new” users as MTServer gets initialized. Multi-Threaded Component Parameter Example Object Manager configuration for 800 Call Center users Concurrent Users 800 Call Center Users Object Manager Level 15% MaxTasks 920 1000 100:1 MaxMTServers 10 MinMTServers 10 Anonymous Users 120 Round up to maintain 100:1 ratio Scaling Across Siebel Servers Depends on how component task initiated Component Group Assignment Which servers a component is available on Object Managers Load balanced (Resonate Central Dispatch) Not valid for eConfigurator Object Manager Siebel Remote Manual allocation of mobile client to Siebel Server Server Requests Server Request Broker &/or Server Request Processor Manual Requests (start task) Manually specified server Special cases CTI eConfigurator Multi-Threaded Component Scalability Vertical Scalability Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Sales Object Manager Siebel Server Enterprise Server Siebel Server Siebel Server Horizontal Scalability Load Balancing Web Client Web Client Web Client Web Client Web Client Web Client Load Balancing Web Server + SWSE Web Server + SWSE Resonate Load Balancing Thread Process Server Sales Object Manager Sales Object Manager Siebel Server Sales Object Manager Sales Object Manager Siebel Server Enterprise Server Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Server Request Broker & Server Request Processor Server Request Broker (SRBroker) Used to start synchronous Siebel Server tasks Server Request Broker & Server Manager are the only components which directly start tasks. New in Siebel 7 Replaces Server Request Manager (SRMSynch) in Siebel 2000 Background component Multi-threaded component Need to set MaxTasks accordingly Server Request Processor (SRProc) Used to start asynchronous Siebel Server tasks Manages queued requests Calls SRBroker to manage task execution Background component Server Request Broker Accepts requests from other Components Will try to service request locally If component is available on same Siebel Server then this is always used If not available locally then will use other Siebel Servers Maintains internal table of components available on each Siebel Server Will route requests round Siebel Servers in turn (round-robin) Multi-threaded component May need to increase MaxTasks Should always be running on all servers Siebel Server – Server Request Broker Assignment Manager Server Request Broker Workflow Process Manager Run Object Assignment Manager Task Is Assignment available on this server? Run Server Server Request Assignment onRequest Broker local server Broker Assignment Manager Workflow Process Manager Siebel Server – Server Request Broker Assignment Manager Run Object Workflow Is Workflow Manager Process Manager Server Request Broker Server Request Broker Workflow Process Manager Assignment Manager available on Server this server? WhichRequest other serversBroker have workflow Choose server online? onWorkflow roundrobin basis Process Manager Server Request Processor Processes asynchronous requests Request submitted by creating record in table S_SRM_REQUEST Server Request Processor reads from table Request must: Be active (reached activation time) Not be specified for a different Siebel Server Not being processed by other Server Request Processor Eligible requests submitted through Server Request Broker Normally runs on all Siebel Servers Siebel Server – Server Request Processor S_SRM_REQUEST SRProc Request Queue SRBroker Task Sleep Interval Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Connection Pooling Siebel 7 can pool sessions at two levels: Web server to Siebel Enterprise SISNAPI Connection Pooling Multiple SISNAPI (Siebel) sessions through one TCP session Reduces operating system overhead and network traffic Enabled by default Set to 20 Controlled by component parameter: ‘Number of Sessions per SISNAPI Connection’ Siebel Object Manager to Database Database connection pooling SQL traffic for multiple Siebel users through one database session Reduces session overheads on database server Disabled by default Suitable for larger deployments (over 500 concurrent users) Database Connection Pooling Connections use native database protocols Some components directly access native protocol Object Managers Siebel 7 supports its own database connection pooling Used for connections from Object Managers Two types of connection Shared – for general transactions Specialized – for long running Not always appropriate Should carefully evaluate tradeoffs Benefits of less database sessions to maintain Risk of database session contention Database Connection Pooling Database session uses login for first user to establish session Database connection pooling controls Defined as component parameters Set to ‘-1’ to disable (default) Specialized (Dedicated) Database sessions All valid per component process (MT Server) per Siebel Server MaxTrxDbConns - Maximum number of specialized DB sessions MinTrxDbConns - Minimum number of specialized DB sessions to be kept in pool Shared Database sessions Valid per component per Siebel Server MaxSharedDbConns- Maximum number of shared DB sessions MinSharedDbConns - Minimum number of shared DB sessions to be kept in pool Database Network Architecture Client Connections Siebel Server Object Manager Server Request Processor Shared Shared Specialized Native Database Connectivity (ODBC for SQL Server) Threads (sessions) Siebel Database Processes (components) Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management General Siebel Server Optimisation Disable unused components and component groups These just use up system resources Consider using common log file for multi-threaded components that generate too many log files Consider using multiple instances/server tasks for certain activities e.g. Parallel EIM, multiple Workflow Policy Groups, etc. Monitor resource utilization by processes and threads Object Manager Optimization Enabling View Caching for High Interactive mode Set in user preferences Faster client response times, by as much as 1 second Better network behavior Expensive, 3MB per view cached Most users use < 10 views / pages Uses LRU mechanism to flush out cache If not using eAdvisor or browser based eConfigurator On cfg file set “EnableCDA = FALSE” Set “Timed Statistics” Off Set all Log Levels to 1 EAI Optimization Minimize fields in Integration Components Keep XML records small XML is very verbose and needs to be parsed Lump several records into one transaction will achieve higher throughput If possible keep records to 15 fields More will reduce throughput Keep in mind that an XML field translates to BC field Integration Object initialize Business Objects and Business Components The more fields per record the slower it will process EAI Optimization Reduce SessPerSisnConn for EAI Adapters Do this only if you are going to “bombard” adapter, i.e. little or no think times For very high throughput systems where you are dedicating EAI machines Consider letting the SRB forward BIM requests to dedicated machines BIM processing can become bottleneck if BIM processing is complex May be necessary to have a higher EAI Adapter:BIM machine ratio (i.e. 1:x) Use ‘Session-Mode’ for high volume inbound HTTP Greatly increased throughput No overhead for authentication & session startup for each message Workflow Process Optimisation Use Run-Time Events for process automation Minimize use of Workflow Policies since workflow policy work at the database layer and do not take advantage of the level of abstraction provided with the Object Manager Performance gain since the decision event takes place on the business object layer and so no trigger is created on the table No need for scripting Workflow Process Distribution Workflow Policies can’t be directly load balanced Asynchronous Workflow Processes are distributed by SRProcessor Synchronous Workflow Process requests are distributed by SRBroker Application Performance - Configuration Keep scripting small and tight Avoid the use of script wherever possible Remember it is still interpreted code and will execute as such Ensure all objects are destroyed after use – avoid memory leaks MVG Applets Keep to a minimum as they increase SQL complexity Use primary joins as much as possible; reduces number of SQL statements Use SmartScript only where really required E.g. Novice Call Center users Avoid putting external news pages on login page 3rd party web sites can be slow Application Performance - Configuration Activate only fields that are absolutely necessary Become active when displayed; “Force Active” in BC configuration General rule, the more fields on a page the slower the page Recently saw customer with 750 fields and 80 tables for one applet Include only manageable set of fields on list applets Many customers use several dozen fields; not usable and slow performance Pages with more than three applets will perform slower for HI applications Reason for this is not really the number of applets, rather the number of fields PickLists set Long List to TRUE if returning large data sets Tree Applets can be slow They generate Bill of Materials type queries Network Performance – Siebel Configuration Browser Validation Reduces the need for server communications to validate data entry Implement through browser script Immediate Posting of Changes Where the ‘Immediate Post Changes’ flag is set against a field data will be transferred whenever a field is changed Incurs additional round trip with approx 2KB data Keep to no more than two Applets per View Minimize Popups Limit columns in List Applets Network Performance – Siebel Settings View Caching View definitions cached in browser memory Requires approx 3MB memory per view Typically around 10 cached views is enough Uses LRU algorithm to maintain cache contents Personalization and Applet Toggles won’t use view caching Enabled through Object Manager configuration (.cfg) file setting [SWE] EnableViewCache=TRUE Controlled through: User Preferences > Behaviour > View Cache Size Default: 10 Network Performance – Siebel Settings Compression (Dynamic Content) Performed by SWSE Do not use web server dynamic compression (application files) Enabled through SWSE configuration file (‘eapps.cfg’) [Defaults] DoCompression = TRUE Compression (Static Content) Performed by web server (IIS only) Network Performance – Web Server / Browser Settings Browser Settings Don’t clear cache except when necessary Ensure ‘Empty Temporary Internet Files Folder when browser is closed’ option is not enabled. Network Performance – Web Server Web Server Settings Use HTTP keep-alive Reduces the need to negotiate TCP sessions for each HTTP message Network Performance – Web Server Web Server Settings Set Content Expiration 2 days is typical setting Set through Internet Information Services Administration HTTP Headers > Content Expiration Network Performance – Web Caching Uncached GET: ‘icon.gif’ 25KB icon.gif DATE: 10/08/03 07:14 RESPONSE: ‘icon.gif’ DATE: 10/10/03 09:25:08 LAST-MODIFIED: 10/08/03 07:14 Network Performance – Web Caching Cached GET: ‘icon.gif’ IF-MODIFIED-SINCE: 10/10/03 09:25 2KB icon.gif icon.gif DATE: 10/10/03 09:25 RESPONSE: Not-modified DATE: 10/08/03 07:14 Network Performance – Web Caching Cached with Expiration 0KB icon.gif icon.gif DATE: 10/10/03 09:25 EXPIRES: 10/12/03 14:13:08 DATE: 10/08/03 07:14 Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Performance – Problem Diagnosis What is the problem? Slow response times reported by User(s)? Resource utilisation problem reported by Administrator? Where is the problem? Who is experiencing the problem? One user, a subset of users, all users? Network problem? Database problem? Siebel Server problem? Web Server problem? How do we resolve the problem? Performance – Problem Diagnosis continued Problem Identification User reports slow response times Identify precise actions carried out by user at the precise time of the problem – set up a problem reporting process, with required information to be supplied by user Subset of Users report slow response times Are users on the same LAN segment? Are other users experiencing problems? Are users all performing similar tasks? All users report slow response times Is it specific transactions (views) that are slow? Do all transactions run slowly? System Administrators should check system resource utilisation System Administrator(s) report resource utilisation problems Are users experiencing problems as a result? Which area of the system is experiencing problems Ask questions of the relevant people – need more information than ‘The system seems to be slow’ Performance – Problem Diagnosis continued Diagnosis Tools Database SQL Trace (SQL Server Profiler) Siebel Server Object Manager SQL Trace Web Server SWSE Statistics Page Web Server SWSE Log Web Server Log (e.g. IIS log) Server Resource Utilisation tools Windows: PerfMon, Task Manager TECHNOTE 361 Memory Utilization in Siebel eBusiness Applications TECHNOTE 382 How can users generate Performance Monitor information into a log file? Performance – SQL Trace Use Database SQL Trace or Siebel Component SQL Trace to identify poor-performing queries Correct Application Configuration Add indexes (except unique indexes) Refresh database statistics (not on Oracle) Siebel Component SQL Trace Set ‘SQL Trace Flags’ parameter Set ‘Event Log Levels’ to 5 for ‘%SQL%’ Srvrmgr: ‘change evtloglvl %SQL%=5 for component sccobjmgr_enu server <siebel server name>’ Scan log file for ‘SQL Statement Execute Time’ Need to use SQL Server Profiler for full capture Siebel uses specialized ODBC connection models Need to reproduce fully in order to get correct query execution plan Performance – Siebel Web Server Logs SWSE Log <Install Dir>/SWEApp/log/ssyymmdd_nn.log Set ‘log=details’ in eapps.cfg file for more detailed logging Set environment variables for full detail SIEBEL_SESSMGR_TRACE=1 SIEBEL_SISNAPI_TRACE=1 SIEBEL_LOGEVENTS=ALL Restart Web Server Web Server Log IIS: <Windows Install Dir>\system32\Logfiles\W3SVC1\exyymmdd.log In Internet Services Manager, enable extended logging to include Client IP address, User Name, Time Taken Use to identify long-running HTTP requests or individual user machines experiencing problems Performance – Siebel SWSE Statistics Page URL: <Application URL>/_stats.swe?verbose=high <Application URL>/_stats.swe?verbose=high&reset=true to reset stats Eapps.cfg configuration file: ‘Allowstats=TRUE’ ‘SessionMonitor=TRUE’ Allows individual user session statistics, in ‘Current Sessions’ section: Event siebel.TCPIP.None.None://jmullisp4:2320/siebel/SCC ObjMgr_enu/jmullisp4/!1.8c0.4023.3ec3708d SADMIN Total Time General Stats (count, mean, std dev) Frequency (mean, std dev) 14.6779 54 0.2718 0.6826 11.33 59 44.64 84 Total Time: Time for all user’s HTTP requests General Stats-Count: # user HTTP requests General Stats-Mean: Average time for each user HTTP request Frequency-Mean: Average time between each user HTTP request Performance – Siebel SWSE Statistics Page continued Session Identification !A.B.C.D A = Siebel Server Id B = Siebel Server Component Process Id C = Siebel Server Component Task Id D = Timestamp All values in hexadecimal Use to map session information to Siebel Server Task logfile Section ‘Current Operations Processing’ Operations in bold type have been running > 10 seconds Volume Performance Testing Using Volume Performance Testing Tools, such as Mercury LoadRunner Segue SilkPerformer Use to measure Client Response times under high user load Resource usage under high user load Web Servers, Siebel Servers, Database Server, Network Scalability of Siebel environment Hardware Use full duplication of production hardware, if possible If not, scale down numbers of servers proportionately Database Tune Object Manager SQL prior to volume performance test exercise Utilise Object Manager SQL trace flags New Feature - SARM Features Siebel Application Response time profiling – SARM (7.5.3) ARM Standard Support (7.7) CPU and Memory Usage Profiling (7.7) Additional Siebel component metrics (7.7) Benefits Proactive monitoring of application response time Diagnosis of response time problems in the application and infrastructure Enables tuning of applications to meet service level commitments Enables diagnosis of memory consumption SARM Run-time Architecture Siebel Server SARM Start Server Component SARM Stop SARM Correlation Key SARM Framework 3rd Party ARM API Library 3rd Party ARM Tool SARM Start SARM Log Server Component SARM Stop ARM Log SARM Instrumentation for 7.5.3 Timers User Interface Physical UI Rendering Web Server Time Logical UI Network Time SWE Time Workflow Time Script Time SRB Time Comp Time Object Manager Data Manager EAI Database Time Siebel Database External Database Legacy Application Performance and Scalability Architecture Overview Component Scalability Scalability Across Components Network Scalability Performance Optimization Performance Management Questions and Answers Siebel 7 Performance and Scalability Inside the Siebel Server Richard Sands Siebel Expert Services ©Siebel Systems 2003 – Do not distribute or re-use without permission