Download Modus v4.4 Database Information

Modus v4.4
Database Information
Database Information
Modus v4.4
Database Information
Latest revision: September 2007
The content of this manual is for informational use only and is subject to change without notice. Neither Vircom nor anyone else who has been involved in the creation or
production of this manual assumes any responsibility or liability for any errors or inaccuracies that may occur in this manual, nor for any loss of anticipated profit or benefits,
resulting from the use of this manual.
This manual is protected by copyright laws and international treaties. Your right to copy this manual is limited by copyright law and the terms of your software license
agreement. As the software licensee, you may make a reasonable number of copies or printouts, provided they are for your own use. Making unauthorized copies, adaptations,
compilations or derivative works for any type of distribution is prohibited and constitutes a punishable violation of the law.
Any references to names of actual companies, products, people and/or data used in screenshots are fictitious and are in no way intended to represent any real individual,
company, product, event and/or data unless otherwise noted.
VOP, Vircom Online Platform, VOP Mail, Modus, ModusMail, ModusGate, Modus Sieve, SCA and VOP Radius are trademarks of Vircom, Inc. Norman Virus Control and NVC
are trademarks of Norman Data Defense Inc. Windows, Windows NT, Windows 2000, Windows Server 2003, IIS, Internet Information Server and Data Access Components
are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Platypus, RODOPI, Emerald, EcoBuilder, Logisense and
Worldgroup are trademarks of their respective owners. All other products or services mentioned in this document are identified by the trademarks or service marks of their
respective companies or organizations.
McAfee and NetScan are registered trademarks of Network Associates, Inc. and/or its affiliates in the US and/or other countries. ©2006 Networks Associates Technology, Inc.
All Rights Reserved.
Portions of this software are based, in part, on ImageMagick, Copyright © 1999-2006, ImageMagick Studio LLC.
ModusMail is based on the Professional Internet Mail Services product licensed from the University of Edinburgh.
Certain algorithms used in parts of this software are derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm.
Copyright © 1995-2007 Vircom, Inc.
Vircom Inc., 460 Ste-Catherine W, Suite 600, Montreal, QC, Canada, H3B 1A7
For more information, contact Customer Support at +1 514.845.8474; Sales at +1.514.845.1666, Ext. 1 or visit our website at www.vircom.com.
2
Database Information
Table of Contents
Introduction ................................................ 4
SQL Server...............................................................................17
Extended (ModusMail only)....................................... 17
Database Formats ...................................... 5
SQL Server...............................................................................17
PostgreSQL ..............................................................................17
Creating a New Database.......................... 6
Security and Maintenance........................18
SQL Server ...................................................................... 6
PostgreSQL ..................................................................... 6
MS SQL Express ............................................................. 6
Running a Database Script ........................ 7
SQL Server ...................................................................... 7
PostgreSQL ..................................................................... 7
MS SQL Express ............................................................. 7
SQL Server Security Enhancements ........................ 18
Maintaining a SQL Server Database......................... 20
Full Recovery Model................................................................20
Simple Recovery Model..........................................................20
Bulk-Logged Recovery Model.................................................21
Changing Recovery Models....................................................21
Optimizing PostgreSQL for Modus.......................... 22
Checkpoint Segments.............................................................22
Max_fsm_pages .....................................................................23
Creating an ODBC Connection ................ 8
SQL Server / SQL Express........................................... 8
PostgreSQL ..................................................................... 8
MS Access...................................................................... 10
Creating and Configuring the Quarantine
Database .................................................... 10
SQL Server / SQL Express......................................... 10
PostgreSQL ................................................................... 10
MS Access...................................................................... 11
Mail Client Quarantine Access ................................. 11
Creating and Configuring the Monitoring
Database .................................................... 11
SQL Server .................................................................... 11
PostgreSQL ................................................................... 12
MS Access...................................................................... 13
Creating and Configuring the Sieve
Database .................................................... 13
SQL Server / SQL Express......................................... 13
PostgreSQL ................................................................... 13
MS Access...................................................................... 14
Creating and Configuring the Audit
Database .................................................... 14
SQL Server / SQL Express......................................... 14
PostgreSQL ................................................................... 15
Creating and Configuring the Extended
Database .................................................... 15
SQL Server .................................................................... 15
PostgreSQL ................................................................... 16
Upgrading the Databases......................... 16
Quarantine..................................................................... 16
SQL Server .............................................................................. 16
PostgreSQL .............................................................................. 16
Audit ............................................................................... 17
3
Database Information
Introduction
This document provides information necessary to create and configure the various
databases required for use by Modus. It also provides the scripts for upgrading
Modus.
This document does not replace the Modus Administration Guides but serves to
compliment the information. Please continue to refer to the Administration Guides
for detailed information concerning your Modus product.
4
Database Information
Database Formats
The following table provides the recommended database formats for Modus:
MS SQL2000+
MS SLQ
Express
PostgreSQL
Quarantine
3
3
3
3
Monitoring
3
3
3
3
Audit
3
3
3
°
Sieve
3
3
3
3
3
3
3
3
Extended
†
•
•
†
& MSDE
ModusMail only
MS Access and MSDE have a size limitation of 2GB
o
These are not recommended for large environments but are acceptable in
small environments
o
Clients should consider using another database program as these have
limitations and will not support future features
SQL Express has a size limitation of 4GB
o
It can be used in medium-sized environments
ƒ
•
MS Access
Database
This only applies to the Audit DB as it can grow fairly large
MySQL can be used for the Quarantine database but is not recommended
o
Additional information can be found in the following KB article:
http://www.vircom.com/kbase/default.asp?id=1328&Lang=1&SID=
5
Database Information
Creating a New Database
The following provides information for creating a new database:
SQL Server
•
Open the SQL Enterprise Manager
•
Expand the tree and go to Databases
•
Click on Action > New Database…
•
In Database Properties, name the database and, at Collation Name, use the dropdown menu to select Latin1_General_CI-AS
o
•
This ensures that the database remains case-insensitive
Click on OK
PostgreSQL
•
Open the pgAdmin III
•
Expand the tree and go to Databases
•
Right-click on Databases and select New Database
•
o
When creating a Quarantine and Sieve databases, select SQL_ASCII for the
encoding
o
For all other databases, select UNICODE for the encoding
Click on OK
MS SQL Express
•
Open the SQL Server Management Studio
•
Right-click Databases and select New Database
•
At the General panel and enter the name of the new database
•
Click on Options and use the drop-down menu to select Latin1_General_CI-AS as
the Collation Name
•
Click on OK
6
Database Information
Running a Database Script
The following provides basic information for running database scripts:
SQL Server
•
Start the SQL Query Analyzer
•
Select the appropriate database
•
Click on File – Open and go to the location of the scripts (…Vircom\Modus<Mail or
Gate>\DBStructures\SQL Server)
•
Click on
f to run the scripts
PostgreSQL
•
Open the pgAdmin III
•
Expand the tree and select the appropriate database
•
Click on Tools > Query Tool
•
Click on File > Open and go to the location of the scripts (…Vircom\Modus<Mail or
Gate>\DBStructures\PostgreSQL)
•
Click on
f to run the scripts
MS SQL Express
•
Open the SQL Server Management Studio
•
Click on File > Open and go to the location of the scripts (…Vircom\Modus<Mail or
Gate>\DBStructures\SQL Server
•
Select the database
•
Click on !Execute to run the scripts
7
Database Information
Creating an ODBC Connection
The following provides information for creating an ODBC connection:
•
On the server, go to Administrative Tools > Data Sources (ODBC)
•
Click on System DSN
•
Click on Add
•
Select the appropriate driver (SQL Server, PostgreSQL or MS Access) and click Finish
SQL Server / SQL Express
•
Enter a name for the data source name
o
Optionally, enter a description of the data source
•
Select the SQL server from the drop-down menu and click Next
•
Select the SQL Server authentication method and enter the login ID and password
•
Click on Next
•
Change the default database to whichever database is being configured (e.g.
Quarantine, Sieve, etc.)
•
Click on Next and then Finish
PostgreSQL
•
Enter a name for the data source name
o
•
Optionally, enter a description of the data source
Enter the name of the database being configured (e.g. Quarantine, Sieve, etc.)
o
Enter the name or IP address of the PostgreSQL server
o
Enter the user name and password
o
Click on Datasource and enter the following information
8
Database Information
o
Go to Page 2 and set the following parameters
o
Click on OK
o
Click on Save
9
Database Information
MS Access
•
Enter the data source name and description
•
Click on Select to chose the database being configured and click on OK
•
Click on OK
•
Exit the ODBC Data Source Administrator panel
Creating and Configuring the Quarantine Database
The following provides information for creating the Quarantine database and configuring it in
Modus:
SQL Server / SQL Express
•
Create a new database
•
Run
the
following
script,
located
in
Gate>\DBStructures\SQL Server\Quarantine
o
…\Vircom\modus<Mail
or
mssql_quarantine_imap.sql
•
Create the ODBC connection
•
In the Modus Console, go to System – Properties – Quarantine Database
•
Enter your ODBC information and click Apply
•
Stop and start all Modus services
PostgreSQL
•
Create a new database
o
•
Remember to select SQL_ASCII for the encoding
Run
the
following
script,
located
in
Gate>\DBStructures\PostgreSQL\Quarantine
o
…\Vircom\modus<Mail
or
pg_quarantine_imap.sql
•
Create the ODBC connection
•
In the Modus Console, go to System – Properties – Quarantine Database
•
Enter your ODBC information and click Apply
•
Stop and start all Modus services
10
Database Information
MS Access
•
The database is created automatically when Modus is installed
•
In the Modus Console, go to System – Properties – Quarantine Database
•
Select Use Native MDB Database
•
Click on Apply
•
Stop and start all Modus services
Mail Client Quarantine Access
If you are using SQL Server or PostgreSQL for the Quarantine database, users can be given
access to their quarantine contents from their mail clients:
ƒ
Ensure that mailboxes are using an IMAP connection to the mail server
•
In the Modus Console, go to System – Properties – Services
o
Select IMAP4S
o
Click Settings
o
IMAP4 Quarantine Access should be checked
•
In the users’ mail client, a Quarantine folder will appear in the folder list along with a
Spam subfolder
•
Only spam messages will appear in the Spam subfolder (not viruses or forbidden
attachments)
•
The contents of the Spam folder are synchronized with WebMail & WebQuarantine
Creating and Configuring the Monitoring Database
The following provides information for creating the Monitoring database and configuring it in
Modus:
SQL Server
•
Create a database
•
Open a Command Prompt on the SQL server
•
Go to …Vircom\Modus<Mail or Gate>\DBStructures\SQL Server\Monitoring
•
Run CreateMonitoringSQL.exe with the following parameters:
createmonitoringsql <server> <database> <username> <password>
•
Create the ODBC connection
11
Database Information
•
In the Modus Console, go to System – Properties – Monitoring Database
•
Select Use ODBC Database and enter your ODBC information
•
Click Apply
•
Stop and start all Modus services
PostgreSQL
If you answered yes to install PostgreSQL during the initial Modus installation, the Monitoring
database was automatically created:
•
In the Modus Console, go to System – Properties – Monitoring Database
•
Select Use Native Postgres Database and enter the appropriate information
•
Click on Apply
•
Stop and start all Modus services
If you answered no to install PostgreSQL during the initial Modus installation ensure that
version 8.0 or higher is installed:
•
•
The necessary files can be found in the …\Vircom\Modus<Mail or Gate> directory:
o
moduspgmon.dll
o
Monitor_pg.sql
Create the Modus tables in your PostgreSQL database and link Modus to it:
o
Extract the moduspgmon.dll file to …\PostgreSQL\8.0\lib
o
From a Command Prompt, go to …\Vircom\Modus<Mail or Gate> and type the
following commands:
SET PGPASSWORD=<<your superuser password>> <enter>
…\Postgresql\8.0\bin\createdb -h localhost -p 5432 -U postgres
-q -E SQL_ASCII -T template1 -O postgres Modus <enter>
…\Postgresql\8.0\bin\psql -h localhost -p 5432 -U postgres -q
-d Modus -f Monitor_pg.sql <enter>
SET PGPASSWORD= <enter>
•
In the Modus Console, go to System – Properties – Monitoring Database
•
Select Use Native Postgres Database and enter the appropriate information
•
Click on Apply
•
Stop and start all Modus services
12
Database Information
MS Access
•
This database is created automatically during the Modus installation
•
In the Modus Console, go to System – Properties – Monitoring Database
•
Select Use Legacy Access Database
•
Click on Apply
•
Stop and start all Modus services
Creating and Configuring the Sieve Database
The following provides information for creating the Sieve database and configuring it in
Modus:
SQL Server / SQL Express
•
Create a new database
•
Run
the
following
script,
located
in
Gate>\DBStructures\SQL Server\SieveStore
o
…\Vircom\modus<Mail
or
mssql_sievestore_new.sql
•
Create the ODBC connection
•
In the Modus Console, go to Spam – Properties – Sieve Database
•
Enter your ODBC information and click Apply
•
Stop and start all Modus services
PostgreSQL
•
Create a new database
o
•
Remember to select SQL_ASCII for the encoding
Run
the
following
script,
located
in
Gate>\DBStructures\PostgreSQL\SieveStore
o
…\Vircom\modus<Mail
or
pg_sievestore_new.sql
•
Create the ODBC connection
•
In the Modus Console, go to Spam – Properties – Sieve Database
•
Enter your ODBC information and click Apply
•
Stop and start all Modus services
13
Database Information
MS Access
•
The database is created automatically when Modus is installed
•
In the Modus Console, go to Spam – Properties – Sieve Database
•
Select Use Native MDB Database
•
Click on Apply
•
Stop and start all Modus services
Creating and Configuring the Audit Database
The following provides information for creating the Message Audit database and configuring it
in Modus:
SQL Server / SQL Express
•
Create a new database
•
Run the following scripts, in this order, located in …\Vircom\modus<Mail or
Gate>\DBStructures\SQL Server\Audit
o
Audit_tables.sql
o
Audit_procs.sql
•
In the Modus Console, go to the System – Properties – Message Audit Database
•
Select Microsoft SQL Server
o
Enter the required information
•
Click on Apply
•
In the Console, go to Logs – Properties – Message Audit
o
Enable Audit Logging and System-Wide Logging
•
Click on Apply
•
Stop and start all Modus services
•
From a Command Prompt, type iisreset <enter> to restart the IIS service
Reminder: if the SQL Server is installed on the same machine as Modus and the Message Audit
is enabled, you will likely experience performance problems.
14
Database Information
PostgreSQL
•
Create a new database
o
•
Remember to select UNICODE for the encoding
Run the following scripts, in this order, located in …\ Vircom\modus<Mail or
Gate>\DBStructures\PostgreSQL\Audit
o
Audit_tables.sql
o
Audit_procs.sql
•
In the Modus Console, go to the System – Properties – Message Audit Database
•
Select PostgreSQL
o
Enter the required information
•
Click on Apply
•
In the Console, go to Logs – Properties – Message Audit
o
Enable Audit Logging and System-Wide Logging
•
Click on Apply
•
Stop and start all Modus services
•
From a Command Prompt, type iisreset <enter> to restart the IIS service
Creating and Configuring the Extended Database
The following provides information for creating the Extended database and configuring it in
Modus. The Extended database is only available for ModusMail.
SQL Server
The following steps must be executed in this order:
•
Create the database
•
Run the following script, located in …Vircom\ModusMail\DBStructures\SQL
Server\ExtendedDB
o
•
mssql_extendeddb_new.sql
Migrate the mailbox settings
SyncUserRegDB.exe tool
o
from
the
registry
to
the
database
the
Select Convert existing registry mailboxes (this is not selected by default)
•
Create an ODBC connection
•
In the Modus Console, go to Auth – Properties – ODBC Database
•
using
o
Ensure Use Extended Database is checked
o
Enter your ODBC information and click Apply
Stop and start all Modus services
15
Database Information
PostgreSQL
The following steps must be executed in this order:
•
Create the database
•
Run
the
following
script,
located
…\Vircom\ModusMail\DBStructures\PostgreSQL\ExtendedDB
o
•
pg_extendeddb_new.sql
Migrate the mailbox settings
SyncUserRegDB.exe tool
o
from
the
registry
to
the
database
using
the
Select Convert existing registry mailboxes (this is not selected by default)
•
Create an ODBC connection
•
In the Modus Console, go to Auth – Properties – ODBC Database
•
in
o
Ensure Use Extended Database is checked
o
Enter your ODBC information and click Apply
Stop and start all Modus services
Upgrading the Databases
The upgrade scripts can be run for whichever version of Modus is being upgraded. Vircom
provides scripts that are run automatically with every upgrade. These scripts are run
manually because they take very long to complete. It is, therefore, recommended that you
run these scripts during off peak times and for every upgrade.
Quarantine
Before running the manual script, stop the MODUSADM service.
SQL Server
Location: …\Vircom\Modus<Mail or Gate>\DBStructures\SQL Server\Quarantine
o
Run mssql_quarantine_manual_upgrade.sql
PostgreSQL
Location: …\Vircom\Modus<Mail or Gate>\DBStructures\PostgreSQL\Quarantine
o
Run pg_quarantine_manual_upgrade.sql
16
Database Information
Audit
SQL Server
Location: …\Vircom\Modus<Mail or Gate>\DBStructures\SQL Server\Audit
o
Stop all Modus services
o
Run audit_upgrade.sql
o
Run audit_procs.sql
o
Start all Modus services
o
Run audit_upgrade_data.sql
Extended (ModusMail only)
Stop all services before running the upgrade. Run the SyncUserRegDB.exe tool (located
with the scripts) and use all defaults. Start the services upon completion.
SQL Server
Location: …\Vircom\Modus<Mail or Gate>\DBStructures\SQL Server\ExtendedDB
o
Run mssql_extendeddb_upgrade.sql
PostgreSQL
Location: …\Vircom\Modus<Mail or Gate>\DBStructures\PostgreSQL\ExtendedDB
o
Run pg_extendeddb_upgrade.sql
17
Database Information
Security and Maintenance
The following provides basic information about the security and maintenance of databases.
These measures are optional but Vircom recommends that you maintain database standards.
SQL Server Security Enhancements
To improve security, the database user should have minimum privileges:
•
No server administrator role
Figure 1: SQL Login Server Roles
•
Access granted to the new database only
Figure 2: SQL Login Database Access
18
Database Information
•
Only db_datareader and db_datawriter permissions for the new database
Figure 3: SQL Database User Role
•
No permission to create, drop or alter tables
•
Permission to execute all DB stored procedures
Figure 4: SQL Database User Permissions
19
Database Information
Maintaining a SQL Server Database
The following provides basic information for maintaining SQL databases.
SQL Server allows for the recovery of databases. The recovery models available for SQL
Server databases each establish the type of backup and restore task that is permitted as well
as the acceptable limit for data loss. There are three recovery models: Full, Simple and Bulk
logged.
Full Recovery Model
•
Uses database and transaction log backups which provide the option for full or
differential restores
•
Allows you to precisely recover the database to a specific time which greatly reduces
the loss of data
•
All operations are logged and can easily be recovered
•
Provides the most flexibility and is the most commonly used option
•
Use this method if:
o
your data is critical and you cannot afford to lose any of it
o
you require recovery to a point in time
o
you use replication and require the option to synchronize all of your databases
to a particular point in time
o
your standard transaction incorporates bulk logged activities
Regular backups are required. It is suggested that a full backup be performed daily and
transaction log backups be performed hourly.
Simple Recovery Model
•
Only restores data from the most recent full or differential backup
•
There are no transaction log backups
o
•
The contents for the log are truncated whenever a checkpoint is issued for the
database
Use this method if:
o
your data is not critical and does not change often
o
data is collected from various sources and can be easily reproduced
o
disk space is limited for logging transaction (the issue with disk space should
be addressed with a possible hardware upgrade)
20
Database Information
Bulk-Logged Recovery Model
•
Provides protection without affecting system performance
•
Some operations are minimally logged and are not fully recoverable
•
Damaged file recovery requires you to manually resolve the issue for operations that
are not completely logged
•
Only allows for database restores to the end of a transaction log backup (where it
contains bulk changes)
•
Use this method if:
o
data is critical but so is system performance
o
bulk operations are performed during off-hours and do not impede regular
processing
o
recovery to a point in time is required
This option is not recommended for use with ModusMail.
Changing Recovery Models
•
In SQL Server, open Enterprise Manager
•
Right click on the database name and select Properties
•
Select the Options tab
•
At Recovery Model use the pull-down menu to select the model
•
Click on OK
Figure 5: Recovery Model Settings
21
Database Information
The recommended minimum settings for Modus databases are:
•
•
•
•
•
Audit: Simple
Quarantine: Simple
Monitoring: Simple
Sieve: Full
Extended: Full
Extended and sieve databases are more important in that they contain the user settings and
some computed values used by the services. As such, a full backup and transaction log
backups will reduce user intervention in the event of a recovery.
Optimizing PostgreSQL for Modus
Checkpoint Segments
Modus data written to the PostgreSQL database is cached in a log file until enough data is
accumulated to warrant a disk-write. Ideally, the log size should be larger than the default
setting as this allows Modus to perform faster (if PostgreSQL writes to the disk too often,
Modus’ performance slows).
The checkpoint_segments configuration setting determines when PostgreSQL can write data
to the disk.
PostgreSQL has two settings for the checkpoints.
The first,
checkpoint_segments, is based on the amount of data modified.
The second,
checkpoint_timeout, is a timeout in seconds. If there are no checkpoints performed in the
past X seconds, PostgreSQL will perform one.
A segment represents a certain amount of data modified. During the PostgreSQL installation,
this value defaults to 3 so that when 3 segments of data are accumulated, PostgreSQL writes
to the disk.
Vircom recommends changing the checkpoint_segments value to 10 for optimum Modus
performance. Keep the default value for checkpoint_timeout.
To change the checkpoint_segments setting:
•
Go to PostgreSQL > Configuration files and select edit postgresql.conf
•
Find the line checkpoint_segments and change the value to 10
•
Remove the # at the start of the line to uncomment the line
•
Save the file
•
Stop and start the POSTGRESQL service on the server
•
Stop and start all Modus services
22
Database Information
Max_fsm_pages
Whenever a database is modified, PostgreSQL allocates new space in pages of 8K. The old
page is still there and cannot be re-used until there is a vacuum command called.
Once every hour, Modus issues the command to be able to re-use the old pages. Re-using an
old page will not increase the size of the file on the disk. The setting max_fsm_pages
indicates the maximum number of pages to mark as re-usable when the vacuum command is
used.
Problems occur when there are more than 20,000 pages that are modified per hour.
Setting the configuration to 1,000,000 pages should re-use all of the old pages even if the
vacuum operation takes many hours to complete. The downside is that it uses more memory.
PostgreSQL will use about 5MG of additional memory when the setting is changed from 20,000
pages.
•
From the Start
postgresql.conf
•
Locate the line #max_fsm_pages
= 1000000
o
menu,
go
to
PostgreSQL
=
>
Configuration
Files
>
Edit
20000 and modify it to max_fsm_pages
Remove the # so that the line is no longer a comment
•
Stop and start the PostgreSQL service
•
Stop and start all Modus services
For more information, consult the Knowledge Base article Info: PostgreSQL Monitoring
Database Grows Too Large.
23