Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
® Conversion/Reversion in IDS Nelson R. Corcoran IBM IBM/Informix DBMS © 2007 IBM Corporation IBM Software Group | Information Management software Objectives of this Talk Learn how conversion/reversion works Learn how to perform conversion/reversion Pitfalls of conversion/reversion Planning for conversion 2 IBM Software Group | Information Management software Outline Why Upgrade Documentation available Migration Paths Planning conversion task – before conversion tasks Straightforward conversion Cautions after conversion Changing page size/OS 32-bit to 64-bit Reversion considerations Convert to Express Edition New features/reserved words in server versions How does it work? 3 IBM Software Group | Information Management software Why upgrade? IDS has been busy detached idxs raw tables buffer manager Fuzzy ckpts HDR/ER Config page size Col Level Encryption large Chunk & files dynamic logging dynamic locking 9.2 update statistics ER Resync optimizer directives onarchive gone 9.3 btree scanners PIT-TLR restartable fast recovery 9.4 Shmem > 4 gig 10.0 IDS Versions – Key features by release 4 IBM Software Group | Information Management software Documentation Ensure that you study the documentation thoroughly !! IBM Informix Migration Guide – primary document Documentation Notes for the IBM Informix Migration Guide Your notes from your last conversion Release notes for the target server 5 IBM Software Group | Information Management software Migration to 10/11 6 IBM Software Group | Information Management software Simple Conversion 1. Install the target database server and copy over relevant $INFORMIXDIR/etc configuration from source to target 2. Validate your environment settings are pointing at target 3. CONVERT by starting target server 4. Validate data integrity and take level 0 backup – DONE! 7 IBM Software Group | Information Management software Planning Conversion Conversion time could be significant – possibly hours – but we don’t modify every page !! Estimate the time required by running some tests on a backup. Test the migration procedures you have. Conversion could fail – you might need to reinstall the old server and restore from Create a “backout plan”. At every step of the migration you should know how to backout and test the backout plan. 8 IBM Software Group | Information Management software Planning Conversion - 2 Conversion across OSs or to a different page size requires a utility driven backup and a restore. This could be time consuming. After conversion, run oncheck. After conversion, update statistics. Consider an “acceptance period”, i.e., no new databases/features used until you are satisfied with the new server. 9 IBM Software Group | Information Management software Planning Conversion - 3 Take performance baseline measurements – compare to new Server. Consider creating or identifying acceptance tests. Severe problems, though somewhat rare, are still quite possible – reversion can be performed after successful conversion. Reversion requires removal of new databases/features. 10 IBM Software Group | Information Management software Before Conversion SQL query plans for important queries (SET EXPLAIN ON) dbschema –d –hd oncheck –pr : dumps reserved pages Copy onconfig file Environment variable list onstat –af, -g all, -g stk all 11 IBM Software Group | Information Management software Before Conversion - 2 During peak times: - onstat –u to see total number of sqlexecs - onstat –p - onstat –g nta - onstat –g nsc, nsd, nss - onstat –P, -g tpf, -g ppf - vmstat, iostat, sar run for 3 to 5 minutes - timex for regularly run queries 12 IBM Software Group | Information Management software Before Conversion - 3 Flush ER Queues with any transactions, if applicable Active HDR servers – disable it, if applicable Level 0 archive – some DBAs take offline OS level backup of chunks or dbexport critical databases Use archecker to verify level 0 archive Shutdown: onmode -sy, onmode -l, onmode -c and then onmode -kuy 13 IBM Software Group | Information Management software Space Considerations Root Chunk (Chunk 0) should have at least 10% free. 3000 Free pages of Logical Log Space to rebuild SysMaster 2000 KB of Free Space per Database is needed for each DB Space Partition Headers should not be full – Use oncheck –me to compress extents 14 IBM Software Group | Information Management software Space Considerations – 2 Determine how much space is needed by running: – DATABASE sysmaster; – – SELECT partdbsnum(partnum) DBSpace_Num, TRUNC(COUNT(*) * 2000) Free_Space_Needed – FROM sysdatabases – GROUP BY 1, ORDER BY 1; – SELECT dbsnum DBSpace_Num, sum(nfree) Spc_Avail – FROM syschunks – GROUP BY 1; 15 IBM Software Group | Information Management software Problem Area: In Place Alters Get rid of these before conversion Do a dummy update, for example: update informix.syscolumns set colno = colno where 1=1; Reversion might fail – especially if you are reverting from 10 + 16 IBM Software Group | Information Management software Starting New Server Don’t do oninit –iy !! Install and configure datablade modules before starting new server Wait for “Conversion Completed Successfully” message to come up Then start verify step 17 IBM Software Group | Information Management software Post Conversion Steps - Conversion of syscdr - Conversion of onpload - Update statistics (high on sysmaster, normal on your databases) - Validate data integrity! - Take level 0 backup - Enable ER/HDR 18 IBM Software Group | Information Management software Post Conversion - 2 For ON-Bar, Rename the sm_versions.std 19 IBM Software Group | Information Management software Page Size/OS Changes Sorry – you have to dump your data!! Use one of the provided utilities: – dbimport/dbexport – load/unload – HPL : ipload/onpladm 20 IBM Software Group | Information Management software 32-bit to 64-bit Good News – works and requires no manual intervention!! We convert some internal data structures from 32-bit to 64 – offsets for example. Remember to change to change SHMBASE and STACKSIZE according to onconfig.std 21 IBM Software Group | Information Management software Reversion Pretty nice – you don’t lose your new data or have to play games unloading/reloading new data! Specify target server for reversion, IDS checks your database first for a number of items then converts it back into the specified target. Reversion might tell you to drop things ie. new databases. You can back them up with utilities first and restore them in the prior version. 22 IBM Software Group | Information Management software Reversion Cautions Things to be cautious with: - large chunks (> 2GB) / new databases - new features such as index self join, truncate table etc. - new reserved words (30 were added in 10.0) - XA data sources and types - secure auditing masks/external spaces - drop triggers and procedures - long identifiers 23 IBM Software Group | Information Management software Reversion Cautions – 2 Stored procedures created or imported with dbimport. In place alters – complete them. Have tables or indexes using expression fragmentation had expressions changed or new fragments added? Have any new new objects and extensions that the old database server does not support been created. Eg: Stprocs(), UDRs, UDTs, etc. 24 IBM Software Group | Information Management software Conversion to Express Edition Normally straightforward, but – No ER – Drop syscdr – No HDR 25 IBM Software Group | Information Management software How Does It Work? Pre 9.4 conversion was a monolithic program. In Version 10 a step wise and partitioned conversion was introduced to improve reliability. Now the server converts from 9.4 to 10 then from 10 to 11.1 Components are ordered: ie. Kernel, SQL layer, SMI, etc. New conversion components can easily be added “in between” existing components 26 IBM Software Group | Information Management software How Does it Work - 2 9.4 Convert to 10 Convert to 11.1 Kernel convert Kernel convert SMI convert SMI convert SQL convert SQL convert Start at the “bottom” and work up 27 IBM Software Group | Information Management software What do we usually change? Usually we add new items to data structures if no “spares” exist. Partition headers are frequently modified. System tables are frequently added or altered with a new column. Add new system databases. Change 32-bit to 64-bit frequently. 28 IBM Software Group | Information Management software Problems Encountered 311 error – can’t open system catalogue – have to restart conversion – caused by applications connecting when the server is not yet converted. This will probably be fixed sometime. 29 IBM Software Group | Information Management software Appendix 30 IBM Software Group | Information Management software Installer for V10 Installer can be invoked by: installserver [preferred] Uses bundled JVM, if JAVA not found in the PATH Installserver –javahome [for users that want to use a local java] Java –cp IIF.jar run [for users who want to invoke directly – advanced] Installserver –legacy [extracts a legacy distribution] Additionally on LINUX also package the following RPMs [since 9.40.UC5] • • • Product.rpm GLS.rpm Message.rpm For enhanced usability, a wrapper script is provided that invokes the RPMs in the correct sequence 31 IBM Software Group | Information Management software ESQL/C – Simply upgrade to the latest version of CSDK and run the application. The new libraries will get loaded providing you enhanced functionality (like encryption). – In order to get new functionality like PAM, applications need to be modified ODBC/.NET/Java – Upgrade to a newer version of CSDK/JDBC driver and run the application 4GL (v 7.32.UC3 ) – c4gl upgrades required applications to be recompiled/relinked. The version 7.32.UC3 removes this requirement. Simply upgrade to this version and run your c4gl applications (supported 7.30.xc6 onwards) . P-code versions still require recompilation. – In order to get new functionality like DYNAMIC ARRAY, applications need to be changed 32 IBM Software Group | Information Management software Environment Variables Verify that the following environment variables are set to the correct values to migrate to the Target server INFORMIXSERVER ONCONFIG PATH INFORMIXSQLHOSTS 33 IBM Software Group | Information Management software Sample Customer Migration 1. Preparation Shutdown User Connections (Dataconsistency) Create and Check last Backup (ontape, dbexport) Increase Parameter LOCKS up to 1'000'000 Increase Physical Log used for conversion and set Environment PLOG_OVERFLOW_PATH oncheck -cDI (each database) oncheck -cr / oncheck -cR oncheck -cc Check installed products (Checks can be made the days before migration) 34 IBM Software Group | Information Management software Sample Customer Migration – 2 2. First Steps Update statistics Low drop Distributions (each database) Remove CDR (cdr delete server) Unregister Datablades from all databases Deactivate your RC-Scripts Make sure all logs have been backed up (no U-----) Change LTAPEDEV in ONCONFIG to /dev/null Shutdown using onmode -ucky Save a copy of „onconfig“ and „sqlhosts“ Backup INFORMIXDIR (for example using tar) 35 IBM Software Group | Information Management software Sample Customer Migration - 3 3. Install new software Check permissions (INFORMIXDIR) Install the new Products (Tools-Engine-SDK) Modify UNIX-Kernel (see Release Notes) Check KAIO-settings (see Release Notes) (for example on HP /etc/privgroup) Make changes in onconfig restore sqlhosts 36 IBM Software Group | Information Management software Sample Customer Migration - 4 4. Start new server oninit -v to see all steps Check online.log to see Conversion and Messages Execute ONMODE -BC 1 and ONMODE -BC 2 (BigChunks) oncheck -cr / oncheck -cc oncheck -cI (each database) oncheck -cD (each database) update statistics (each database) update statistics high (on leading indexcolumns) 37 IBM Software Group | Information Management software Sample Customer Migration - 5 5. Backup and Documentation Take the first backup of the new version Initiate CDR Register Datablades Activate RC-Scripts Make a snapshot of your installation (onstat -a) 38