Download Arbiter documentation

Arbiter documentation
Install
Table of Contents
Install Arbiter Database ................................................................................................. 2
LogMiner ........................................................................................................................... 3
Using Abakus LogMiner ................................................................................................. 3
Using Internal Oracle LogMiner ..................................................................................... 3
Using External Oracle LogMiner .................................................................................... 3
Install GUI & Tools .......................................................................................................... 4
Configure PHP ................................................................................................................ 5
Confrigure Nginx ............................................................................................................ 6
Configure url rewriting ............................................................................................... 6
Restrict Access ........................................................................................................... 6
Configure HTTP Push Module .................................................................................... 6
Optional: Configure SSL ............................................................................................. 6
Optional: Guidelines ................................................................................................... 7
Configure Comet ............................................................................................................. 8
Configure Oracle Wallet ............................................................................................. 8
nginx Warnings ............................................................................................................... 9
PHP Warnings ............................................................................................................. 9
Example Windows Service Wrapper configuration .................................................... 9
nginx ........................................................................................................................... 9
PHP ............................................................................................................................. 9
Arbiter GUI Config .................................................................................................... 10
Source Database Configuration .................................................................................. 11
Oracle ........................................................................................................................... 12
Enable Archive Mode ................................................................................................ 12
Enable Force Logging ............................................................................................... 12
Enable Auditing ........................................................................................................ 12
Oracle Reference ...................................................................................................... 13
Minimal Supplemental Logging ............................................................................ 13
MS SQL Server ............................................................................................................. 14
Prerequisites ............................................................................................................. 14
Create new Windows user ........................................................................................ 14
Create SQL Server user ............................................................................................ 14
Configure SQL Server ............................................................................................... 14
Setup Certificates ..................................................................................................... 14
Install & configure Window Service ......................................................................... 15
Rsyslog .......................................................................................................................... 17
Install Arbiter Database
●
Create (empty) Oracle Database 11g Release 2 database any way you like, just make
sure following requirements are met:
You can optionally run create_database.sh to automate database creation if you want.
Following components/options are installed
XDB: Oracle XML Database (@?/rdbms/admin/catqm.sql)
JAVAVM: JServer JAVA Virtual Machine (@?/javavm/install/initjvm.sql)
CATJAVA: Oracle Database Java Packages (@?/rdbms/admin/catjava.sql)
Following tablespaces exists (Arbiter has hardcoded names of these tablespaces):
TEMP (temporary tablespace)
USERS (every auditor (arbiter user) has this as default tablespace)
TS_AUDIT (if not specified otherwise, this is default tablespace for metadata
schemas REV, REV_LOGMNR and REV_TPL)
Run install.sql script (located in ./dbsource/1.3/ folder of your Arbiter installation)
Optionally follow steps described in configuring Oracle source database if you wish to
register loopback (Arbiter database itself).
Default user created after clean install is rev_admin/rev_admin1#.
If you're using RAC, make sure logminer is using logs from ASM or from cluster disk (so
that both instances can access the same files).
❍
❍
■
■
■
❍
■
■
■
●
●
●
●
Arbiter documentation - Install
2
LogMiner
Using Abakus LogMiner
Setting
Value
job_d_get_implementation ALP
Using Internal Oracle LogMiner
Setting
Value
job_d_get_implementation INTERNAL
This is the simplest way, just set it to INTERNAL and Arbiter database will parse archive
logs using its Oracle Logminer API. You cannot have more than one database set to
INTERNAL, because Oracle does not support it.
Using External Oracle LogMiner
Setting
Value
job_d_get_implementation EXTERNAL
external_logmnr_database TNS name of LogMiner database
If you have more than one Oracle source database registered, you need to create one
logminer database for each additional source database. This is because Oracle Logminer
cannot run multiple logminers at the same time. This can be done in following steps:
●
●
●
●
●
●
Create new oracle database instance
create REV_LOGMNR user
run logminer.sql
create database link to Arbiter database
run rev_logmnr.ga_data_oracle_logminer.create_logminer_job()
lock the REV_LOGMNR user account.
Arbiter documentation - Install
3
Install GUI & Tools
●
●
●
●
Install latest stable release of PHP 5.3 (Windows Guide)
Install latest stable release of nginx with HTTP Push Module (Windows Guide)
Configure nginx virtual host
Create configuration file (copy admin/Setts.php.def to admin/Setts.php)
Edit CONF_DATABASE setting to match TNS name of Arbiter database to connect to.
Other settings should work fine with default values most of the time.
Set folders permissions by running reset_permissions.sh script
Configure GUI Connectivity (at minimum, set GUI URL parameter in Administration ->
Settings -> GUI Connectivity -> GUI URL)
❍
●
●
Arbiter documentation - Install
4
Configure PHP
Default php.ini should work.
Arbiter documentation - Install
5
Confrigure Nginx
Configure url rewriting
location / {
if (!-e $request_filename) {
rewrite ^(.*)$ /index.php?q= last;
}
}
Restrict Access
location
location
location
location
^~
^~
^~
^~
/admin/
/tools/
/dbsource/
/languages/
{deny
{deny
{deny
{deny
all;
all;
all;
all;
break;}
break;}
break;}
break;}
Configure HTTP Push Module
location /comet {
push_channel_group push_arbiter;
location /comet/pub {
set $push_channel_id arbiter;
push_publisher;
push_message_timeout 5s;
push_message_buffer_length 10;
}
location /comet/sub {
set $push_channel_id arbiter;
push_subscriber;
send_timeout 3600;
}
}
Optional: Configure SSL
ssl on;
ssl_certificate /opt/arbiter/admin/cert/arbiter-gui.pem;
ssl_certificate_key /opt/arbiter/admin/cert/arbiter-gui.pem;
location ~ \.php {
# make sure $_SERVER['https'] is set
fastcgi_param HTTPS on;
}
Arbiter documentation - Install
6
Self-signed certificates can be generated using makecert.sh utility.
Optional: Guidelines
●
●
Use /opt/arbiter as Arbiter root directory
Enable SSL and use admin/cert/arbiter_gui.pem as certificate file name for GUI
Arbiter documentation - Install
7
Configure Comet
Log into GUI and go to Administration > Settings > GUI Connectivity
●
●
●
GUI URL is most likely what you see in address bar in your browser, but database must
be able to resolve this.
Oracle Wallet Path if using SSL, this is where Oracle finds public key for gui.
Oracle Wallet Pass if using SSL, this is the password of Oracle Wallet where the public
key of the GUI is stored.
Configure Oracle Wallet
This example assumes you don't yet have a suitable Oracle Wallet configured. If you used
makecert utility to genereate your self-signed certificate (public key) then public key
(certificate) is already located in admin/cert/cert.pub.crt.
$ mkdir /home/oracle/wallet/
$ orapki wallet create -wallet /home/oracle/wallet/arbiter.wallet -pwd
"arbiter123" -auto_login
$ orapki wallet add -wallet /home/oracle/wallet/arbiter.wallet -trusted_cert
-cert /opt/arbiter/admin/cert/arbiter-gui.pub.crt -pwd "arbiter123"
Arbiter documentation - Install
8
nginx Warnings
●
●
●
●
●
Using nginx for Windows disables Arbiter's progress bar feature.
Make sure that the path you extracted (=installed) to do not include any spaces. Use,
for example C:\opt\nginx, not "Program Files"
Windows tend to append suffixes and then don't display it (by default), so if you saved
test.html, make sure it is not really named test.html.txt.
In nginx.conf file use forward slashes (/) instead of backslashes (\) for path separators.
Double backslash may work sometimes, but the actual path then becomes something like
C:\path_to\/my_file.html.
After the service is created set its startup type to Automatic (Delayed Start)
PHP Warnings
●
●
●
Builds from http://windows.php.net are 32 bit, so database or instant client must also be
32 bit.
For PHP to work, you need Microsoft Visual C++ 2008 Redistributable Package
(x86) (not 2010, but 2008), available from http://download.microsoft.com/
After the service is created set its startup type to Automatic (Delayed Start)
Example Windows Service Wrapper configuration
nginx
●
●
Just notice the -p base_path parameter, otherwise it defaults to C:\windows\system32
from where the service is started.
Any slash will do :)
<service>
<id>nginx</id>
<name>nginx</name>
<description>nginx HTTP Server</description>
<executable>C:\opt\nginx\nginx.exe</executable>
<logpath>C:\opt\service_wrapper\logs\</logpath>
<startargument>-p C:\opt\nginx</startargument>
<stopargument>-p C:\opt\nginx -s stop</stopargument>
</service>
PHP
●
●
Don't forget PATH env, otherwise OCI extension won't load
also set -c path_to\php.ini, by default it searches for .ini in C:\Windows\
<service>
<id>php</id>
<name>php</name>
<description>PHP Hypertext Preprocessor - CGI service</description>
<executable>C:\opt\php\php-cgi.exe</executable>
<logpath>C:\opt\service_wrapper\logs\</logpath>
Arbiter documentation - Install
9
<startargument>-b 127.0.0.1:9000 -c C:\opt\php\php.ini</startargument>
<env name="PATH" value="C:\opt\instantclient_11_2\;%PATH%" />
<env name="TNS_ADMIN" value="C:\opt\instantclient_11_2\conf\" />
<env name="ORACLE_HOME" value="C:\opt\instantclient_11_2\" />
<env name="NLS_LANG" value="AMERICAN_AMERICA.AL32UTF8" />
</service>
Arbiter GUI Config
Since Windows build of nginx does not support comet, GUI should not use this
functionality. Add this to admin/Setts.php:
define('CONF_JOBS_AUTOSTART', false);
Arbiter documentation - Install
10
Source Database Configuration
List of supported data sources:
●
●
●
●
Oracle Database 11g
Oracle Database 10g
Microsoft SQL Server 2008
rsyslog
Arbiter documentation - Install
11
Oracle
Enable Archive Mode
Source database should be in archivelog mode. This is required if you wish to track data
changes (old/new values). Archives should be located in folder defined by
log_archive_dest_1. Make sure you type the trailing / as some versions of Oracle
Database contain bug, which prevents writing to the archive log files if there is no trailing
/ character!
startup mount;
alter system set log_archive_dest_1 = 'LOCATION=+DATA/mydb/arch/';
alter database archivelog;
alter database open;
Make sure that archive logs are not deleted before Arbiter fetches them. Backup should
delete only logs older than 2 days
RMAN Example:
RMAN> backup archivelog until time 'sysdate - 2' delete input;
Enable Force Logging
Arbiter gets data from archive log files, so we should make sure that every transaction is
logged:
ALTER DATABASE FORCE LOGGING;
Enable Auditing
Auditing should be enabled, you can do that by running:
alter
alter
alter
alter
system
system
system
system
set
set
set
set
audit_trail = 'db','extended' scope = spfile;
audit_sys_operations=true scope = spfile;
audit_file_dest='/path/to/audit/' scope = spfile;
audit_syslog_level='local0.info' scope = spfile; --optional
You may omit 'extended' keyword, if you don't want SQL query text to be logged. xml, db
and os are all accaptable values for Arbiter. We recommend to use XML (minimal
overhead with ability to log sys actions), but the choice is yours (consult Oracle Database
official documentation)
Set audit_sys_operations to true if you would like to track actions taken by the DBA staff.
Arbiter documentation - Install
12
Set audit_file_dest for location of your audit files
●
●
●
audit_trail='db': only dba actions are written here in .aud files
audit_trail='xml': all actions are written here in .xml files
audit_trail='os': all actions are written here in .aud files
Set audit_syslog_level if you wish to send audit files to your syslog. Arbiter can get audit
entries from there too.
●
●
audit_trail='os': all audit actions are written to the syslog.
audit_traiL='db': only dba actions are written to syslog.
Oracle Reference
●
●
●
●
audit_trail parameter
audit_sys_operations
audit_file_dest
audit_syslog_level
Minimal Supplemental Logging
You may want to restart your database for this operation, otherwise it may take very (very)
long to complete. This is a required setting even if you don't want to track data changes
(old/new values)
alter database add supplemental log data;
Arbiter documentation - Install
13
MS SQL Server
Prerequisites
●
●
●
Install .NET Framework 3.5 SP1 if not already installed.
Install Microsoft® SQL Server® 2008 R2 Shared Management Objects (Part of MS SQL
Server Feature Pack)
Install Oracle Client (latest patchset is available at Oracle Support)
Unzip downloaded file, run setup.exe
Select Installation Type: custom
Available Product Components: .NET and OLE items.
❍
❍
❍
Create new Windows user
●
●
Start > Administrative Tools > Computer Management
System Tools > Local Users and Groups > Users. Right click, "New User..."
Do not add it to Users group - this group has interactive logon privilege by default
Set username like AuditCollector. Don't forget to set password too.
Start > Administrative Tools > Local Security Policy
Local Policies > User Rights Assignment: Add AuditCollector to Logon as a service
❍
●
●
❍
Grant user privilege to use following URL for serving SOAP requests by using netsh (or
httpcfg.exe on Win 2003 and older):
netsh http add urlacl url=https://+:8501/ArbiterService
user=WINSQL\AuditCollector
Create SQL Server user
You can run script like this or do a few clicks around the SQL Server Management Studio:
CREATE LOGIN [WINSQL\AuditCollector] FROM WINDOWS;
GO
USE [ProductionDatabase];
CREATE USER [AuditCollector] FOR LOGIN [WINSQL\AuditCollector];
EXEC sp_addrolemember N'db_owner', N'AuditCollector';
GO
Configure SQL Server
No special configuration is needed, just make sure SQL Server Agent is running if you
are planning to use CDC (tracking old/new values). You can do that by changing startup
type from manual (default) to automatic (Windows Services).
Setup Certificates
Communication between Arbiter and MS SQL Collector service is encrypted. Also, both,
Arbiter and service must authenticate each other.
To generate new certificate, you can use our wrapper for openssl (makecert.sh) available
as an Arbiter command line tool. Alternatively, you can use any tool you like, such as
Arbiter documentation - Install
14
makecert (part of Windows SDK).
On the source side, you'll need public key (let's call it arbiter.crt) from your Arbiter
installation (it should already be in your Oracle Database Wallet if you configured Arbiter
according to our guidelines) and your private+public key of [newly] generated certificate let call that one winsql.pfx.
●
●
Run Microsoft Management Console (Start > Run: mmc)
File > Add/Remove Snap-in...
Certificates, Computer Account, Local Computer
Finish, OK
Right click on Personal (those are certificate with which service can introduce itself), All
Tasks, Import.
Find and choose winsql.pfx file. Next, next, ... Finish
Right click on Trusted People (those are certificates to whom you trust to be ok)
Import arbiter.crt the same way you did the winsql.pfx (you should not include private
key thogh)
After successfull import it will appear on the right side, right click > Open > Details
Select Thumbprint (Show All) and write put it in AuditTrailCollector.exe.config
(applicationSettings>setting[name=AllowedClientCertificates]). Careful if you use
copy/paste, the first character will be "invisible" and collector will refuse the
connection. Safe bet is to retype it manually. Do not include any space characters.
Now that you trust that certificate arbiter.crt is valid, you should also trust to whoever
issued it (CA Root Certificate). For self-signed certificates with no real CA, you can use
that same arbiter.crt file and import it to Trusted Root Certification Authorities
(same way as you did for Personal and Trusted).
❍
❍
●
❍
●
❍
❍
■
●
Finally, allow service to use certificate on port on which service will be bound:
netsh http add sslcert ipport=0.0.0.0:8501
certhash=d8891bf1e3d27d4efdeeddf583e9190341ec27a9
appid={B49A0F8E-F7B7-427d-91E2-5EC2B951DC4B} clientcertnegotiation=enable
Install & configure Window Service
●
●
●
Unzip CollectorService-ver.zip to a folder like C:\Program Files\Arbiter
Make sure AuditCollector (Windows user which will run this service) has all permissions
for this folder.
Configure logging by editing Log4Net.config (this is actuall a XML file)
Edit name of the log file (<file value="C:\Program
Files\Arbiter\logs\AuditTrailCollector.log")
Configure collector settings in settings.xml:
DatabaseConnectionString is connection string of the MS SQL Database (set to
value like Data Source=WINSQL;Initial Catalog=master;Integrated Security = true)
StorageDatabaseConnectionString is connection string of Arbiter (Oracle)
Database (set to value like Data Source=ARBITER;User Id=rev_collector
;Password=rev_collector;Integrated Security=no;)
StorageFilePath is folder where audit data is put if connection to Arbiter could not
be established. Data is transferred to Arbiter when connection comes back online.
Make sure this folder exits. If string is not provided, data generated while Arbiter is
offline is lost.
StorageErrorPath is folder where audit data which resulted any kind of error is put.
❍
●
❍
❍
❍
❍
Arbiter documentation - Install
15
●
Perhaps something collector could not parse. If string is not provided, data which
resulted an error is lost (error is logged though).
Configure AuditTrailCollectorService.exec.config
Edit data source name of target production database <connectionStrings><add
connectionString="Data Source=WINSQL; Initial Catalog=master; Integrated
Security=True"
You can set alternative port and hostname for SOAP service to listen on.
baseAddress=server.your.domain.com:8501 (make sure Arbiter database can
resolve and access this).
❍
❍
Finally create Windows service using sc.exe (run as Administrator):
sc create AuditTrailCollectorService binpath= "C:\Program
Files\Arbiter\AuditTrailCollector.exe" start= auto obj=
"WINSQL\AuditCollector" password= abcd123 DisplayName= "Arbiter audit trail
collector service"
Arbiter documentation - Install
16
Rsyslog
This is sample configuration of rsyslog.conf
$ModLoad omoracle
$OmoracleDBUser rev_collector
$OmoracleDBPassword collector
$OmoracleDB arbiter.abakus.si
$OmoracleBatchSize 1
$OmoracleBatchItemSize 4096
$OmoracleStatementTemplate OmoracleStatement
$template OmoracleStatement,"INSERT INTO trail_1234(rev_aud_id,
hostname,ts,hostip,facility,severity,program,message) VALUES
(trail_sq_1234.nextval, :hostname,to_timestamp_tz(:dategen || ' ' || :timegen,
'YYYY-MM-DD
HH24:MI:SS.FF6TZH:TZM'),:hostip,:facility,:severity,:program,:message)"
$template
TestStmt,"%hostname%%timereported:0:10:date-rfc3339%%timereported:12:32:daterfc3339%%fromhost-ip%%syslogfacility%%syslogseverity%%programname%%msg%"
*.*
:omoracle:;TestStmt
Note the number 1234 in the above example, this is database id. Once registered you
can find it in GUI.
Arbiter documentation - Install
17
Arbiter documentation - Install
18