* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Arbiter documentation
Survey
Document related concepts
Microsoft Access wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Functional Database Model wikipedia , lookup
Concurrency control wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Relational model wikipedia , lookup
Database model wikipedia , lookup
ContactPoint wikipedia , lookup
Transcript
Arbiter documentation Install Table of Contents Install Arbiter Database ................................................................................................. 2 LogMiner ........................................................................................................................... 3 Using Abakus LogMiner ................................................................................................. 3 Using Internal Oracle LogMiner ..................................................................................... 3 Using External Oracle LogMiner .................................................................................... 3 Install GUI & Tools .......................................................................................................... 4 Configure PHP ................................................................................................................ 5 Confrigure Nginx ............................................................................................................ 6 Configure url rewriting ............................................................................................... 6 Restrict Access ........................................................................................................... 6 Configure HTTP Push Module .................................................................................... 6 Optional: Configure SSL ............................................................................................. 6 Optional: Guidelines ................................................................................................... 7 Configure Comet ............................................................................................................. 8 Configure Oracle Wallet ............................................................................................. 8 nginx Warnings ............................................................................................................... 9 PHP Warnings ............................................................................................................. 9 Example Windows Service Wrapper configuration .................................................... 9 nginx ........................................................................................................................... 9 PHP ............................................................................................................................. 9 Arbiter GUI Config .................................................................................................... 10 Source Database Configuration .................................................................................. 11 Oracle ........................................................................................................................... 12 Enable Archive Mode ................................................................................................ 12 Enable Force Logging ............................................................................................... 12 Enable Auditing ........................................................................................................ 12 Oracle Reference ...................................................................................................... 13 Minimal Supplemental Logging ............................................................................ 13 MS SQL Server ............................................................................................................. 14 Prerequisites ............................................................................................................. 14 Create new Windows user ........................................................................................ 14 Create SQL Server user ............................................................................................ 14 Configure SQL Server ............................................................................................... 14 Setup Certificates ..................................................................................................... 14 Install & configure Window Service ......................................................................... 15 Rsyslog .......................................................................................................................... 17 Install Arbiter Database ● Create (empty) Oracle Database 11g Release 2 database any way you like, just make sure following requirements are met: You can optionally run create_database.sh to automate database creation if you want. Following components/options are installed XDB: Oracle XML Database (@?/rdbms/admin/catqm.sql) JAVAVM: JServer JAVA Virtual Machine (@?/javavm/install/initjvm.sql) CATJAVA: Oracle Database Java Packages (@?/rdbms/admin/catjava.sql) Following tablespaces exists (Arbiter has hardcoded names of these tablespaces): TEMP (temporary tablespace) USERS (every auditor (arbiter user) has this as default tablespace) TS_AUDIT (if not specified otherwise, this is default tablespace for metadata schemas REV, REV_LOGMNR and REV_TPL) Run install.sql script (located in ./dbsource/1.3/ folder of your Arbiter installation) Optionally follow steps described in configuring Oracle source database if you wish to register loopback (Arbiter database itself). Default user created after clean install is rev_admin/rev_admin1#. If you're using RAC, make sure logminer is using logs from ASM or from cluster disk (so that both instances can access the same files). ❍ ❍ ■ ■ ■ ❍ ■ ■ ■ ● ● ● ● Arbiter documentation - Install 2 LogMiner Using Abakus LogMiner Setting Value job_d_get_implementation ALP Using Internal Oracle LogMiner Setting Value job_d_get_implementation INTERNAL This is the simplest way, just set it to INTERNAL and Arbiter database will parse archive logs using its Oracle Logminer API. You cannot have more than one database set to INTERNAL, because Oracle does not support it. Using External Oracle LogMiner Setting Value job_d_get_implementation EXTERNAL external_logmnr_database TNS name of LogMiner database If you have more than one Oracle source database registered, you need to create one logminer database for each additional source database. This is because Oracle Logminer cannot run multiple logminers at the same time. This can be done in following steps: ● ● ● ● ● ● Create new oracle database instance create REV_LOGMNR user run logminer.sql create database link to Arbiter database run rev_logmnr.ga_data_oracle_logminer.create_logminer_job() lock the REV_LOGMNR user account. Arbiter documentation - Install 3 Install GUI & Tools ● ● ● ● Install latest stable release of PHP 5.3 (Windows Guide) Install latest stable release of nginx with HTTP Push Module (Windows Guide) Configure nginx virtual host Create configuration file (copy admin/Setts.php.def to admin/Setts.php) Edit CONF_DATABASE setting to match TNS name of Arbiter database to connect to. Other settings should work fine with default values most of the time. Set folders permissions by running reset_permissions.sh script Configure GUI Connectivity (at minimum, set GUI URL parameter in Administration -> Settings -> GUI Connectivity -> GUI URL) ❍ ● ● Arbiter documentation - Install 4 Configure PHP Default php.ini should work. Arbiter documentation - Install 5 Confrigure Nginx Configure url rewriting location / { if (!-e $request_filename) { rewrite ^(.*)$ /index.php?q= last; } } Restrict Access location location location location ^~ ^~ ^~ ^~ /admin/ /tools/ /dbsource/ /languages/ {deny {deny {deny {deny all; all; all; all; break;} break;} break;} break;} Configure HTTP Push Module location /comet { push_channel_group push_arbiter; location /comet/pub { set $push_channel_id arbiter; push_publisher; push_message_timeout 5s; push_message_buffer_length 10; } location /comet/sub { set $push_channel_id arbiter; push_subscriber; send_timeout 3600; } } Optional: Configure SSL ssl on; ssl_certificate /opt/arbiter/admin/cert/arbiter-gui.pem; ssl_certificate_key /opt/arbiter/admin/cert/arbiter-gui.pem; location ~ \.php { # make sure $_SERVER['https'] is set fastcgi_param HTTPS on; } Arbiter documentation - Install 6 Self-signed certificates can be generated using makecert.sh utility. Optional: Guidelines ● ● Use /opt/arbiter as Arbiter root directory Enable SSL and use admin/cert/arbiter_gui.pem as certificate file name for GUI Arbiter documentation - Install 7 Configure Comet Log into GUI and go to Administration > Settings > GUI Connectivity ● ● ● GUI URL is most likely what you see in address bar in your browser, but database must be able to resolve this. Oracle Wallet Path if using SSL, this is where Oracle finds public key for gui. Oracle Wallet Pass if using SSL, this is the password of Oracle Wallet where the public key of the GUI is stored. Configure Oracle Wallet This example assumes you don't yet have a suitable Oracle Wallet configured. If you used makecert utility to genereate your self-signed certificate (public key) then public key (certificate) is already located in admin/cert/cert.pub.crt. $ mkdir /home/oracle/wallet/ $ orapki wallet create -wallet /home/oracle/wallet/arbiter.wallet -pwd "arbiter123" -auto_login $ orapki wallet add -wallet /home/oracle/wallet/arbiter.wallet -trusted_cert -cert /opt/arbiter/admin/cert/arbiter-gui.pub.crt -pwd "arbiter123" Arbiter documentation - Install 8 nginx Warnings ● ● ● ● ● Using nginx for Windows disables Arbiter's progress bar feature. Make sure that the path you extracted (=installed) to do not include any spaces. Use, for example C:\opt\nginx, not "Program Files" Windows tend to append suffixes and then don't display it (by default), so if you saved test.html, make sure it is not really named test.html.txt. In nginx.conf file use forward slashes (/) instead of backslashes (\) for path separators. Double backslash may work sometimes, but the actual path then becomes something like C:\path_to\/my_file.html. After the service is created set its startup type to Automatic (Delayed Start) PHP Warnings ● ● ● Builds from http://windows.php.net are 32 bit, so database or instant client must also be 32 bit. For PHP to work, you need Microsoft Visual C++ 2008 Redistributable Package (x86) (not 2010, but 2008), available from http://download.microsoft.com/ After the service is created set its startup type to Automatic (Delayed Start) Example Windows Service Wrapper configuration nginx ● ● Just notice the -p base_path parameter, otherwise it defaults to C:\windows\system32 from where the service is started. Any slash will do :) <service> <id>nginx</id> <name>nginx</name> <description>nginx HTTP Server</description> <executable>C:\opt\nginx\nginx.exe</executable> <logpath>C:\opt\service_wrapper\logs\</logpath> <startargument>-p C:\opt\nginx</startargument> <stopargument>-p C:\opt\nginx -s stop</stopargument> </service> PHP ● ● Don't forget PATH env, otherwise OCI extension won't load also set -c path_to\php.ini, by default it searches for .ini in C:\Windows\ <service> <id>php</id> <name>php</name> <description>PHP Hypertext Preprocessor - CGI service</description> <executable>C:\opt\php\php-cgi.exe</executable> <logpath>C:\opt\service_wrapper\logs\</logpath> Arbiter documentation - Install 9 <startargument>-b 127.0.0.1:9000 -c C:\opt\php\php.ini</startargument> <env name="PATH" value="C:\opt\instantclient_11_2\;%PATH%" /> <env name="TNS_ADMIN" value="C:\opt\instantclient_11_2\conf\" /> <env name="ORACLE_HOME" value="C:\opt\instantclient_11_2\" /> <env name="NLS_LANG" value="AMERICAN_AMERICA.AL32UTF8" /> </service> Arbiter GUI Config Since Windows build of nginx does not support comet, GUI should not use this functionality. Add this to admin/Setts.php: define('CONF_JOBS_AUTOSTART', false); Arbiter documentation - Install 10 Source Database Configuration List of supported data sources: ● ● ● ● Oracle Database 11g Oracle Database 10g Microsoft SQL Server 2008 rsyslog Arbiter documentation - Install 11 Oracle Enable Archive Mode Source database should be in archivelog mode. This is required if you wish to track data changes (old/new values). Archives should be located in folder defined by log_archive_dest_1. Make sure you type the trailing / as some versions of Oracle Database contain bug, which prevents writing to the archive log files if there is no trailing / character! startup mount; alter system set log_archive_dest_1 = 'LOCATION=+DATA/mydb/arch/'; alter database archivelog; alter database open; Make sure that archive logs are not deleted before Arbiter fetches them. Backup should delete only logs older than 2 days RMAN Example: RMAN> backup archivelog until time 'sysdate - 2' delete input; Enable Force Logging Arbiter gets data from archive log files, so we should make sure that every transaction is logged: ALTER DATABASE FORCE LOGGING; Enable Auditing Auditing should be enabled, you can do that by running: alter alter alter alter system system system system set set set set audit_trail = 'db','extended' scope = spfile; audit_sys_operations=true scope = spfile; audit_file_dest='/path/to/audit/' scope = spfile; audit_syslog_level='local0.info' scope = spfile; --optional You may omit 'extended' keyword, if you don't want SQL query text to be logged. xml, db and os are all accaptable values for Arbiter. We recommend to use XML (minimal overhead with ability to log sys actions), but the choice is yours (consult Oracle Database official documentation) Set audit_sys_operations to true if you would like to track actions taken by the DBA staff. Arbiter documentation - Install 12 Set audit_file_dest for location of your audit files ● ● ● audit_trail='db': only dba actions are written here in .aud files audit_trail='xml': all actions are written here in .xml files audit_trail='os': all actions are written here in .aud files Set audit_syslog_level if you wish to send audit files to your syslog. Arbiter can get audit entries from there too. ● ● audit_trail='os': all audit actions are written to the syslog. audit_traiL='db': only dba actions are written to syslog. Oracle Reference ● ● ● ● audit_trail parameter audit_sys_operations audit_file_dest audit_syslog_level Minimal Supplemental Logging You may want to restart your database for this operation, otherwise it may take very (very) long to complete. This is a required setting even if you don't want to track data changes (old/new values) alter database add supplemental log data; Arbiter documentation - Install 13 MS SQL Server Prerequisites ● ● ● Install .NET Framework 3.5 SP1 if not already installed. Install Microsoft® SQL Server® 2008 R2 Shared Management Objects (Part of MS SQL Server Feature Pack) Install Oracle Client (latest patchset is available at Oracle Support) Unzip downloaded file, run setup.exe Select Installation Type: custom Available Product Components: .NET and OLE items. ❍ ❍ ❍ Create new Windows user ● ● Start > Administrative Tools > Computer Management System Tools > Local Users and Groups > Users. Right click, "New User..." Do not add it to Users group - this group has interactive logon privilege by default Set username like AuditCollector. Don't forget to set password too. Start > Administrative Tools > Local Security Policy Local Policies > User Rights Assignment: Add AuditCollector to Logon as a service ❍ ● ● ❍ Grant user privilege to use following URL for serving SOAP requests by using netsh (or httpcfg.exe on Win 2003 and older): netsh http add urlacl url=https://+:8501/ArbiterService user=WINSQL\AuditCollector Create SQL Server user You can run script like this or do a few clicks around the SQL Server Management Studio: CREATE LOGIN [WINSQL\AuditCollector] FROM WINDOWS; GO USE [ProductionDatabase]; CREATE USER [AuditCollector] FOR LOGIN [WINSQL\AuditCollector]; EXEC sp_addrolemember N'db_owner', N'AuditCollector'; GO Configure SQL Server No special configuration is needed, just make sure SQL Server Agent is running if you are planning to use CDC (tracking old/new values). You can do that by changing startup type from manual (default) to automatic (Windows Services). Setup Certificates Communication between Arbiter and MS SQL Collector service is encrypted. Also, both, Arbiter and service must authenticate each other. To generate new certificate, you can use our wrapper for openssl (makecert.sh) available as an Arbiter command line tool. Alternatively, you can use any tool you like, such as Arbiter documentation - Install 14 makecert (part of Windows SDK). On the source side, you'll need public key (let's call it arbiter.crt) from your Arbiter installation (it should already be in your Oracle Database Wallet if you configured Arbiter according to our guidelines) and your private+public key of [newly] generated certificate let call that one winsql.pfx. ● ● Run Microsoft Management Console (Start > Run: mmc) File > Add/Remove Snap-in... Certificates, Computer Account, Local Computer Finish, OK Right click on Personal (those are certificate with which service can introduce itself), All Tasks, Import. Find and choose winsql.pfx file. Next, next, ... Finish Right click on Trusted People (those are certificates to whom you trust to be ok) Import arbiter.crt the same way you did the winsql.pfx (you should not include private key thogh) After successfull import it will appear on the right side, right click > Open > Details Select Thumbprint (Show All) and write put it in AuditTrailCollector.exe.config (applicationSettings>setting[name=AllowedClientCertificates]). Careful if you use copy/paste, the first character will be "invisible" and collector will refuse the connection. Safe bet is to retype it manually. Do not include any space characters. Now that you trust that certificate arbiter.crt is valid, you should also trust to whoever issued it (CA Root Certificate). For self-signed certificates with no real CA, you can use that same arbiter.crt file and import it to Trusted Root Certification Authorities (same way as you did for Personal and Trusted). ❍ ❍ ● ❍ ● ❍ ❍ ■ ● Finally, allow service to use certificate on port on which service will be bound: netsh http add sslcert ipport=0.0.0.0:8501 certhash=d8891bf1e3d27d4efdeeddf583e9190341ec27a9 appid={B49A0F8E-F7B7-427d-91E2-5EC2B951DC4B} clientcertnegotiation=enable Install & configure Window Service ● ● ● Unzip CollectorService-ver.zip to a folder like C:\Program Files\Arbiter Make sure AuditCollector (Windows user which will run this service) has all permissions for this folder. Configure logging by editing Log4Net.config (this is actuall a XML file) Edit name of the log file (<file value="C:\Program Files\Arbiter\logs\AuditTrailCollector.log") Configure collector settings in settings.xml: DatabaseConnectionString is connection string of the MS SQL Database (set to value like Data Source=WINSQL;Initial Catalog=master;Integrated Security = true) StorageDatabaseConnectionString is connection string of Arbiter (Oracle) Database (set to value like Data Source=ARBITER;User Id=rev_collector ;Password=rev_collector;Integrated Security=no;) StorageFilePath is folder where audit data is put if connection to Arbiter could not be established. Data is transferred to Arbiter when connection comes back online. Make sure this folder exits. If string is not provided, data generated while Arbiter is offline is lost. StorageErrorPath is folder where audit data which resulted any kind of error is put. ❍ ● ❍ ❍ ❍ ❍ Arbiter documentation - Install 15 ● Perhaps something collector could not parse. If string is not provided, data which resulted an error is lost (error is logged though). Configure AuditTrailCollectorService.exec.config Edit data source name of target production database <connectionStrings><add connectionString="Data Source=WINSQL; Initial Catalog=master; Integrated Security=True" You can set alternative port and hostname for SOAP service to listen on. baseAddress=server.your.domain.com:8501 (make sure Arbiter database can resolve and access this). ❍ ❍ Finally create Windows service using sc.exe (run as Administrator): sc create AuditTrailCollectorService binpath= "C:\Program Files\Arbiter\AuditTrailCollector.exe" start= auto obj= "WINSQL\AuditCollector" password= abcd123 DisplayName= "Arbiter audit trail collector service" Arbiter documentation - Install 16 Rsyslog This is sample configuration of rsyslog.conf $ModLoad omoracle $OmoracleDBUser rev_collector $OmoracleDBPassword collector $OmoracleDB arbiter.abakus.si $OmoracleBatchSize 1 $OmoracleBatchItemSize 4096 $OmoracleStatementTemplate OmoracleStatement $template OmoracleStatement,"INSERT INTO trail_1234(rev_aud_id, hostname,ts,hostip,facility,severity,program,message) VALUES (trail_sq_1234.nextval, :hostname,to_timestamp_tz(:dategen || ' ' || :timegen, 'YYYY-MM-DD HH24:MI:SS.FF6TZH:TZM'),:hostip,:facility,:severity,:program,:message)" $template TestStmt,"%hostname%%timereported:0:10:date-rfc3339%%timereported:12:32:daterfc3339%%fromhost-ip%%syslogfacility%%syslogseverity%%programname%%msg%" *.* :omoracle:;TestStmt Note the number 1234 in the above example, this is database id. Once registered you can find it in GUI. Arbiter documentation - Install 17 Arbiter documentation - Install 18