Download Information Management Software Support Update

®
Conversion/Reversion in IDS
Nelson R. Corcoran
IBM
IBM/Informix DBMS
© 2007 IBM Corporation
IBM Software Group | Information Management software
Objectives of this Talk
 Learn how conversion/reversion works
 Learn how to perform conversion/reversion
 Pitfalls of conversion/reversion
 Planning for conversion
2
IBM Software Group | Information Management software
Outline

Why Upgrade

Documentation available

Migration Paths

Planning conversion task – before conversion tasks

Straightforward conversion

Cautions after conversion

Changing page size/OS

32-bit to 64-bit

Reversion considerations

Convert to Express Edition

New features/reserved words in server versions

How does it work?
3
IBM Software Group | Information Management software
Why upgrade? IDS has been busy
detached
idxs
raw
tables
buffer
manager
Fuzzy
ckpts
HDR/ER
Config page size
Col Level
Encryption
large
Chunk
& files
dynamic
logging
dynamic
locking
9.2
update
statistics
ER Resync
optimizer
directives
onarchive
gone
9.3
btree
scanners
PIT-TLR
restartable
fast recovery
9.4
Shmem
> 4 gig
10.0
IDS Versions – Key features by release
4
IBM Software Group | Information Management software
Documentation
 Ensure that you study the documentation thoroughly !!
 IBM Informix Migration Guide – primary document
 Documentation Notes for the IBM Informix Migration Guide
 Your notes from your last conversion
 Release notes for the target server
5
IBM Software Group | Information Management software
Migration to 10/11
6
IBM Software Group | Information Management software
Simple Conversion
1. Install the target database server and copy over relevant
$INFORMIXDIR/etc configuration from source to target
2. Validate your environment settings are pointing at target
3. CONVERT by starting target server
4. Validate data integrity and take level 0 backup – DONE!
7
IBM Software Group | Information Management software
Planning Conversion
 Conversion time could be significant – possibly hours – but we
don’t modify every page !!
 Estimate the time required by running some tests on a backup.
 Test the migration procedures you have.
 Conversion could fail – you might need to reinstall the old server
and restore from
 Create a “backout plan”. At every step of the migration you
should know how to backout and test the backout plan.
8
IBM Software Group | Information Management software
Planning Conversion - 2
 Conversion across OSs or to a different page size requires a
utility driven backup and a restore. This could be time
consuming.
 After conversion, run oncheck.
 After conversion, update statistics.
 Consider an “acceptance period”, i.e., no new
databases/features used until you are satisfied with the new
server.
9
IBM Software Group | Information Management software
Planning Conversion - 3
 Take performance baseline measurements – compare to new
Server.
 Consider creating or identifying acceptance tests.
 Severe problems, though somewhat rare, are still quite possible –
reversion can be performed after successful conversion.
 Reversion requires removal of new databases/features.
10
IBM Software Group | Information Management software
Before Conversion
 SQL query plans for important queries (SET EXPLAIN ON)
 dbschema –d –hd
 oncheck –pr : dumps reserved pages
 Copy onconfig file
 Environment variable list
 onstat –af, -g all, -g stk all
11
IBM Software Group | Information Management software
Before Conversion - 2
 During peak times:
- onstat –u to see total number of sqlexecs
- onstat –p
- onstat –g nta
- onstat –g nsc, nsd, nss
- onstat –P, -g tpf, -g ppf
- vmstat, iostat, sar run for 3 to 5 minutes
- timex for regularly run queries
12
IBM Software Group | Information Management software
Before Conversion - 3
 Flush ER Queues with any transactions, if applicable
 Active HDR servers – disable it, if applicable
 Level 0 archive – some DBAs take offline OS level backup of
chunks or dbexport critical databases
 Use archecker to verify level 0 archive
 Shutdown: onmode -sy, onmode -l, onmode -c and then onmode
-kuy
13
IBM Software Group | Information Management software
Space Considerations
 Root Chunk (Chunk 0) should have at least 10% free.
 3000 Free pages of Logical Log Space to rebuild SysMaster
 2000 KB of Free Space per Database is needed for each DB
Space
 Partition Headers should not be full
– Use oncheck –me to compress extents
14
IBM Software Group | Information Management software
Space Considerations – 2
 Determine how much space is needed by running:
– DATABASE sysmaster;
–
–
SELECT partdbsnum(partnum) DBSpace_Num,
TRUNC(COUNT(*) * 2000) Free_Space_Needed
–
FROM sysdatabases
–
GROUP BY 1, ORDER BY 1;
–
SELECT dbsnum DBSpace_Num, sum(nfree) Spc_Avail
–
FROM syschunks
–
GROUP BY 1;
15
IBM Software Group | Information Management software
Problem Area: In Place Alters
 Get rid of these before conversion
 Do a dummy update, for example:
 update informix.syscolumns set colno = colno where 1=1;
 Reversion might fail – especially if you are reverting from 10 +
16
IBM Software Group | Information Management software
Starting New Server
 Don’t do oninit –iy !!
 Install and configure datablade modules before starting new
server
 Wait for “Conversion Completed Successfully” message to come
up
 Then start verify step
17
IBM Software Group | Information Management software
Post Conversion Steps
- Conversion of syscdr
- Conversion of onpload
- Update statistics (high on sysmaster, normal on your databases)
- Validate data integrity!
- Take level 0 backup
- Enable ER/HDR
18
IBM Software Group | Information Management software
Post Conversion - 2
 For ON-Bar, Rename the sm_versions.std
19
IBM Software Group | Information Management software
Page Size/OS Changes
 Sorry – you have to dump your data!!
 Use one of the provided utilities:
– dbimport/dbexport
– load/unload
– HPL : ipload/onpladm
20
IBM Software Group | Information Management software
32-bit to 64-bit
 Good News – works and requires no manual intervention!!
 We convert some internal data structures from 32-bit to 64 –
offsets for example.
 Remember to change to change SHMBASE and STACKSIZE
according to onconfig.std
21
IBM Software Group | Information Management software
Reversion
 Pretty nice – you don’t lose your new data or have to play games
unloading/reloading new data!
 Specify target server for reversion, IDS checks your database
first for a number of items then converts it back into the specified
target.
 Reversion might tell you to drop things ie. new databases. You
can back them up with utilities first and restore them in the prior
version.
22
IBM Software Group | Information Management software
Reversion Cautions
Things to be cautious with:
- large chunks (> 2GB) / new databases
- new features such as index self join, truncate table etc.
- new reserved words (30 were added in 10.0)
- XA data sources and types
- secure auditing masks/external spaces
- drop triggers and procedures
- long identifiers
23
IBM Software Group | Information Management software
Reversion Cautions – 2
 Stored procedures created or imported with dbimport.
 In place alters – complete them.
 Have tables or indexes using expression fragmentation had
expressions changed or new fragments added?
 Have any new new objects and extensions that the old database
server does not support been created. Eg: Stprocs(), UDRs,
UDTs, etc.
24
IBM Software Group | Information Management software
Conversion to Express Edition
 Normally straightforward, but
– No ER
– Drop syscdr
– No HDR
25
IBM Software Group | Information Management software
How Does It Work?
 Pre 9.4 conversion was a monolithic program.
 In Version 10 a step wise and partitioned conversion was
introduced to improve reliability.
 Now the server converts from 9.4 to 10 then from 10 to 11.1
 Components are ordered: ie. Kernel, SQL layer, SMI, etc.
 New conversion components can easily be added “in between”
existing components
26
IBM Software Group | Information Management software
How Does it Work - 2
9.4
Convert to 10
Convert to 11.1
Kernel convert
Kernel convert
SMI convert
SMI convert
SQL convert
SQL convert
Start at the “bottom” and work up
27
IBM Software Group | Information Management software
What do we usually change?
 Usually we add new items to data structures if no “spares” exist.
 Partition headers are frequently modified.
 System tables are frequently added or altered with a new
column.
 Add new system databases.
 Change 32-bit to 64-bit frequently.
28
IBM Software Group | Information Management software
Problems Encountered
 311 error – can’t open system catalogue – have to restart
conversion – caused by applications connecting when the
server is not yet converted. This will probably be fixed
sometime.
29
IBM Software Group | Information Management software
Appendix
30
IBM Software Group | Information Management software
Installer for V10

Installer can be invoked by:

installserver [preferred] Uses bundled JVM, if JAVA not found in the PATH

Installserver –javahome [for users that want to use a local java]

Java –cp IIF.jar run [for users who want to invoke directly – advanced]

Installserver –legacy [extracts a legacy distribution]

Additionally on LINUX also package the following RPMs [since 9.40.UC5]
•
•
•

Product.rpm
GLS.rpm
Message.rpm
For enhanced usability, a wrapper script is provided that invokes the RPMs in the correct sequence
31
IBM Software Group | Information Management software
ESQL/C
– Simply upgrade to the latest version of CSDK and run the application. The new
libraries will get loaded providing you enhanced functionality (like encryption).
– In order to get new functionality like PAM, applications need to be modified
ODBC/.NET/Java
– Upgrade to a newer version of CSDK/JDBC driver and run the application
4GL (v 7.32.UC3 )
– c4gl upgrades required applications to be recompiled/relinked. The version
7.32.UC3 removes this requirement. Simply upgrade to this version and run your
c4gl applications (supported 7.30.xc6 onwards) . P-code versions still require
recompilation.
– In order to get new functionality like DYNAMIC ARRAY, applications need to be
changed
32
IBM Software Group | Information Management software
Environment Variables
Verify that the following environment variables are set to the correct
values to migrate to the Target server
INFORMIXSERVER
ONCONFIG
PATH
INFORMIXSQLHOSTS
33
IBM Software Group | Information Management software
Sample Customer Migration
1. Preparation
Shutdown User Connections (Dataconsistency)
Create and Check last Backup (ontape, dbexport)
Increase Parameter LOCKS up to 1'000'000
Increase Physical Log used for conversion and
set Environment PLOG_OVERFLOW_PATH
oncheck -cDI (each database)
oncheck -cr / oncheck -cR
oncheck -cc
Check installed products

(Checks can be made the days before migration)
34
IBM Software Group | Information Management software
Sample Customer Migration – 2
2. First Steps
Update statistics Low drop Distributions (each database)
Remove CDR (cdr delete server)
Unregister Datablades from all databases
Deactivate your RC-Scripts
Make sure all logs have been backed up (no U-----)
Change LTAPEDEV in ONCONFIG to /dev/null
Shutdown using onmode -ucky
Save a copy of „onconfig“ and „sqlhosts“
Backup INFORMIXDIR (for example using tar)
35
IBM Software Group | Information Management software
Sample Customer Migration - 3
3. Install new software
Check permissions (INFORMIXDIR)
Install the new Products (Tools-Engine-SDK)
Modify UNIX-Kernel (see Release Notes)
Check KAIO-settings (see Release Notes)
(for example on HP /etc/privgroup)
Make changes in onconfig
restore sqlhosts
36
IBM Software Group | Information Management software
Sample Customer Migration - 4
4. Start new server
oninit -v to see all steps
Check online.log to see Conversion and Messages
Execute ONMODE -BC 1 and ONMODE -BC 2 (BigChunks)
oncheck -cr / oncheck -cc
oncheck -cI (each database)
oncheck -cD (each database)
update statistics (each database)
update statistics high (on leading indexcolumns)
37
IBM Software Group | Information Management software
Sample Customer Migration - 5
5. Backup and Documentation
Take the first backup of the new version
Initiate CDR
Register Datablades
Activate RC-Scripts
Make a snapshot of your installation (onstat -a)
38