Download HP TRIM Sql Server – Setup and Maintenance

HP TRIM Sql Server – Setup and Maintenance
Author: Ashley Bill
Date: June 2010
Version: 1.0
1
HP TRIM Sql Server – Setup and Maintenance
Introduction
The purpose of this document is to outline best practice as to setting up a SQL Server environment for
HP TRIM Context. Most of this information comes from best practice, known issues and experience
from performance tuning in the field so there will be many references to Sql Server 2005. Sql Server
2008 is supported by TRIM 6.2.4 and TRIM 7 upwards. At the moment, there are currently no
additional optimizations or ‘gotchas’ for this particular release beyond those of Sql server 2005.
This document will cover all the basics of database setup and its on-going maintenance. Use this
document by first reading the chapter on the TRIM pre-requisites and then by dipping in and out when
configuring different database components.
Contents
HP TRIM Sql Server – Setup and Maintenance ......................................................................................... 1
Introduction ............................................................................................................................................. 2
Contents .................................................................................................................................................. 2
TRIM pre-requisites .................................................................................................................................... 3
Versions .................................................................................................................................................. 3
Character sets .......................................................................................................................................... 3
Compatibility Mode ................................................................................................................................ 3
RDBMS client 64-bit support ................................................................................................................. 3
Database Creation ................................................................................................................................... 3
File Layout .............................................................................................................................................. 4
Filegroups ........................................................................................................................................... 4
File Growth ......................................................................................................................................... 4
Initial Size ........................................................................................................................................... 5
Auto-extend......................................................................................................................................... 5
Unrestricted Growth............................................................................................................................ 5
Log Files ............................................................................................................................................. 5
Manually purging and shrinking the log files ..................................................................................... 5
Create a trim login, User and Schema. ....................................................................................................... 6
Create a Maintenance Plan for the Database. ............................................................................................. 6
TOP PRIORITY - Rebuild Index Task ................................................................................................... 6
THE OTHER TOP PRIORITY - Update Statistics Task ....................................................................... 6
Backups ................................................................................................................................................... 6
History Clean-up ..................................................................................................................................... 6
Maintenance Cleanup Task ..................................................................................................................... 6
Setup Alerts Using the SQL Server Agent ................................................................................................. 6
Database Properties ..................................................................................................................................... 7
Auto_Create_Statistics ............................................................................................................................ 7
Auto_Update_Statistics........................................................................................................................... 7
Locking Level ......................................................................................................................................... 7
Setting the Server-level Values. .................................................................................................................. 8
Memory ................................................................................................................................................... 8
Advanced – cursor threshold limit. ......................................................................................................... 8
Annex A – Additional Statistics for poorly performing searches ............................................................... 9
Introduction ............................................................................................................................................. 9
Workaround / Fix .................................................................................................................................... 9
2
HP TRIM Sql Server – Setup and Maintenance
TRIM pre-requisites
Versions
TRIM is compatible with SQL Server 2005 and Sql Server 2008 (versions of TRIM 6.2.4 and TRIM 7
upwards only for 2008). The minimum service packs for each version of RDBMS are specified in the
installation documentation, but as a rule of thumb, apply all the latest RDBMS service packs. Sql Server
2000 is basically very old. Some versions of TRIM still support it but it is now best to not use Sql
Server 2000 with TRIM any longer.
Character sets
The database character set must be configured to use Dictionary Order, Case Insensitive sort order. The
Accent Insensitive option is also recommended, but is not mandatory. TRIM Context cannot be
configured for, or operate on, a SQL Server database binary or case sensitive sort order.
Latin1_general_CI_AI – Recommended
Latin1_general_CI_AS – Alternative.
Compatibility Mode
The install documentation says to have a minimum compatibility mode of 70, preferably 80. Personally
I would keep this as high as possible in-line with the database version.
70 = SQL Server 7.0
80 = SQL Server 2000
90 = SQL Server 2005
100 = SQL Server 2008
RDBMS client 64-bit support
This is straight from the installation document: Whether you use a 32-bit or 64-bit client depends on
whether your HP TRIM application is 32-bit or 64-bit. A 64-bit RDBMS client can only be used with a
64-bit application. Similarly, a 32-bit client can only be used with a 32-bit application. Check the
version of HP TRIM you are installing and install the corresponding RDBMS client. It does not matter
whether the database you are connecting to is 32-bit or 64-bit.
Database Creation
In past versions of TRIM, you could use the TRIM Enterprise Manager to create the SQL Server
database and dataset/schema. This is useful for a quick installation in a development or R&D but not
good for production. From TRIM 7 onwards you must now create the database manually.
Using the ‘sa’ user for everything will work but takes no account of security. Creating the database
using the defaults will work also but will not help performance. To this end, the following work flow
3
HP TRIM Sql Server – Setup and Maintenance
should be used when creating a SQL Server 2005 database to support TRIM in environments other than
development:




Design and setup the storage solution for the database datafiles.
Create the User and Database in the SQL Server Management Studio (SSMS).
o Login using the ‘sa’ account.
o Create the database manually.
 Create separate filegroups for data and indexes.
 Put the files in the correct locations.
 Separate files for the data and indexes if possible - (for backup and recovery
reasons).
o Create a trim login, user and schema.
Create a maintenance plan for the database.
Setup alerts using the SQL Server agent.
Next, install the TRIM dataset as per normal, ensuring you use the TRIM user and the correct filegroups.
Information to support the above process is contained in the remainder of this document. Additional
information is included in this document as a simple aid-memoir.
File Layout
In the perfect world all files would each have their own separate disk. Financial constraints and SAN
storage means this never happens. RAID and SAN mean that most of the time everything supporting the
database goes on the SAN or disk array.
There will be times when this cannot happen so you need to know, given a set of disks which files can
reside where. Firstly, to reduce contention, all data files should not be on the same disk(s) as the O/S or
RDBMS binaries. So you should at least have a C and D drive.
Each database has 2 types of files - data and log files. The data files hold application information. The
log files hold information on the database activity. It is best to put each on a separate disk as each needs
to be written to simultaneously.
Filegroups
Many datafiles can be collected together in filegroups. An example of a filegroup is the PRIMARY
(default) filegroup. This filegroup holds the system tables for the database. Microsoft recommends
creating a separate filegroup specifically for applications – this reduces contention with system tables.
To this end, I would suggest creating a TRIM_DATA as a filegroup. You are best to spread all the files
in a filegroup throughout the available disks to increase I/O throughput. You might even decide to
create a separate group of files for indexes. Realistically, this may not help on performance (especially
with a SAN or RAID array) but would certainly help with backup and recovery options. Here, the more
files, the less chance of a single corrupt file bringing down the whole database.
File Growth
When you create a file keep the following in-mind:
4
HP TRIM Sql Server – Setup and Maintenance
Initial Size
Try to set its initial size to that close to what you think you will need. This will avoid time delays
caused by a file extending automatically.
Auto-extend
You can use auto extend especially as most of the time you will not know exactly how large the file will
get. Try and auto extend in large chunks so that a file will not have to extend too frequently. Avoid
percentage increase growth. This could suddenly hit you one day. e.g. A growth of 10% of a 100M file
is small and accommodatable. If the file gets to 1Gb or even 10G, all of a sudden you will have files
extending by HUGE chunks (it happens believe me)!
Unrestricted Growth
Be sure you know what you are doing here before setting this value. There is the risk that you may end
up with a huge unmanageable file. Set a limit for all files – something manageable say 5-10Gb. Space
requirements are linear. The following is just a guide as the size really depends upon the types and sizes
of files added by users:
Number of records
10,000
100,000
1,000,000
Size of file (Mb)
50Mb
500Mb
5Gb
Log Files
There is a log file for each database. It records the history of what happened on the database to support
read consistency and recovery. As it records everything, it can get big very quickly unless maintained.
You can manually truncate a log file and then manually shrink it if it gets too big. This will remove any
old data in the file with regards to transactions which have been committed to disk. The reason why this
is not done automatically is that the act of truncating this data, with no backup, removes your chance to
‘play-back’ the history of changes in the event of a database recovery. This would probably be ok for
many environments but certainly not for production. To this end, Sql Server 2005 provides a database
‘Recovery Model’.
Non-production environments - A recovery model with value ‘simple’ means that the log file will be
automatically, kept free of old data. However, you must still remember to shrink it manually.
Production environments - A recovery model of ‘full’ means that by use of backups, the log file will be
purged of old data but only once it has been backed up. Again you must manually shrink the file.
The manual file shrinking can be done through use of a maintenance job.
Manually purging and shrinking the log files
To shrink the log file you can either use the SSMS or the T-SQL command:
5
HP TRIM Sql Server – Setup and Maintenance
DBCC SHRINKFILE ('<logfile_name>' , 0, TRUNCATEONLY)
Create a trim Login, User and Schema.
1.
2.
3.
4.
Create a login for the TRIM user - give this login the public role and use SQL Server
authentication.
Create a database schema named TRIM.
Create a database user named TRIM and associate it with the login and schema above.
Grant db_ddladmin to the TRIM user.
Use this login to create the trim dataset and not the sa login
Create a Maintenance Plan for the Database.
Setup the following maintenance activities:
TOP PRIORITY - Rebuild Index Task
Set to work out-of-hours, at least once a week. This task will rebuild all database indexes and update
their statistics.
THE OTHER TOP PRIORITY - Update Statistics Task
Update all existing statistics at least once a week preferably daily if the system is under high load. Use
the setting ‘full scan’ if performance problems occur. If performance problems persist then follow the
directions at Annex A.
Backups
THINK. You backup the database and the external document store – how will recovery of these be
coordinated? If you choose point-in-time recovery ‘full’ backup mode then this would be the best as the
External Document Store backups will probably vary and be out of your control. However, if you
choose full backup mode then remember to have a job which shrinks the log file (This is a good article).
History Clean-up
Remove historical data as required.
Maintenance Cleanup Task
Remove old backup files as required
Setup Alerts Using the SQL Server Agent
The SQL Server agent runs maintenance jobs. Additionally, it can be used to monitor server activity in
the form of alerts. Setup alerts to cover severity levels 17 and above. Also setup alerts for deadlocks.
6
HP TRIM Sql Server – Setup and Maintenance
Database Properties
Auto_Create_Statistics
When auto_create_statistics is turned on (which it is by default), statistics are automatically created on
all columns used in the WHERE clause of a query. This occurs when a query is optimized by the Query
Optimizer for the first time, assuming the column doesn't already have statistics created for it. The
addition of column statistics can greatly aid the Query Optimizer so that it can help create an optimum
execution plan for the query. If this option is turned off, then missing column statistics are not
automatically created, which can mean that the Query Optimizer may not be able to produce the
optimum execution plan for the query, and the query's performance may suffer.
Auto_Update_Statistics
In order for the Query Optimizer to make smart query optimization decisions, column and index
statistics need to be up-to-date. The best way to ensure this is to leave the auto_update_statistics
database option on (the default setting). This helps to ensure that the optimizer statistics are valid,
helping to ensure that queries are properly optimized when they are run. But this option is not a panacea.
When a SQL Server database is under very heavy load, sometimes the auto_update_statistics feature can
update the statistics on large tables at inappropriate times, such as the busiest time of the day. If you find
that the auto_update_statistics feature is running at inappropriate times, you may want to turn it off, and
then manually update the statistics. But again, consider what will happen if you do turn off the
auto_update_statistics feature! To this end we recommend setting auto_update_statistics on
supplemented by a maintenance task which regularly (nightly) updates statistics.
Locking Level
7
HP TRIM Sql Server – Setup and Maintenance
TRIM has very few issues with deadlocks. One customer did have a problem with Sql Server where the
default locking setting was not correct. This is not a recommendation but advice. If, and only if,
deadlocks appear in the system try altering the default locking setting to
READ_COMMITTED_SNAPSHOT.
Setting the Server-level Values.
The server-level value default settings will work fine. Only change the following settings if
performance issues occur or when going through testing to get optimum performance.
Memory
Leave as default unless you want to cap the maximum amount used by the database server. On a 32bit
installation, the maximum amount of memory available for use to the RDBMS is 4Gb. This should be
enough for most but the largest installations of TRIM. If you need to go bigger then you will need to
configure the RDBMS and server to use AWE to allocate memory.
Advanced – cursor threshold limit.
Leave this setting unless trying to look at a performance problem. The cursor threshold limit is used to
indicate when the datasets supporting cursors should be generated synchronously or asynchronously.
Setting this to a number greater than say 1000 may improve performance for querys returning >1000
records.
8
HP TRIM Sql Server – Setup and Maintenance
Annex A – Additional Statistics for poorly performing searches
Introduction
There can be circumstances where the amount and type of data in the database can lead to the RDBMS
Optimizer selecting a slow Plan for searches. To manage this situation, addition statistics need to be
taken. You know when this will be required because execution plans have ‘Lazy Spools’ listed in them.
Workaround / Fix
Do the following daily using a maintenance plan:
1.
2.
3.
All database indexes should be rebuilt.
Full RDBMS table scan statistics were taken on all database objects.
Statistics taken on TSRECWOR and TSINDEXWOR to include ‘WITH INDEX’.
UPDATE STATISTICS TSRECWORD WITH INDEX
UPDATE STATISTICS TSINDEXWOR WITH INDEX
4.
Create the following index:
CREATE NONCLUSTERED INDEX ix_accessco1 ON TSACCESSCO (axObjectUri
ASC, axObject ASC, axType ASC, axLocUri ASC)
9
HP TRIM Sql Server – Setup and Maintenance