Download P8CE-3.5.2-018 CBR indexing changes

P8CE-3.5.2-018 CBR indexing changes
Validating Indexed Content and Properties
Purpose:
A new command-line application – CEIxCheck.exe - is installed with P8CE-3.5.2-018.
This application can be used to validate entries in a set of CBR indexes.
The tool allows:
1) Determine which documents have not been indexed.
2) Deleting duplicate indexes.
3) Re-indexing documents.
The application can be run while the Content Engine is in use although it is better to run it during low
load times.
Usage:
Check indexed File Store content:
CEIxCheck -check ObjectStoreName FileStoreName
[-date "YYYY-MM-DD [hh:mm]"]
[-bc] [-noquery] [-datapath <path>]
[-reindex] [-delete]
[-u[e] username password/encrpyted password]
Check indexed database content and/or properties:
CEIxCheck -check ObjectStoreName [-props] [-content]
[-date "YYYY-MM-DD [hh:mm]"]
[-bc] [-noquery] [-datapath <path>]
[-reindex] [-delete]
[-u[e] username password/encrpyted password]
Run “CEIxCheck.exe” with no command line arguments to display the above
help information.
Required permissions
If the application is not run as a P8 domain administrator then use the "u[e]" option to connect to the
Object Store as a domain administrator.
The user running the application must have read permissions on the File Store index directories being
validated.
The user must be able to write to the current directory or the path specified with the "datapath" option.
When using reindex or delete without the bc option, run the application as the Object Store service user
(FNCE_<machine>). When running without the bc option, the application re-indexes by creating CE
index requests with the Object Store service.
If File Store content is being checked with the reindex option then bc should not be used as nothing will
be re-indexed.
Usage
To validate a File Store's content indexes:
CEIxCheck -check ObjectStoreName FileStoreName
To validate an Objects Store's database content and/or CBR-indexed properties (at least one of the
options must be specified):
CEIxCheck -check ObjectStoreName [-props] [-content]
datapath:
File path for data files created by the application.
Uses the current directory if not specified.
noquery:
Do not run database queries – use the data files created from a previous run.
date:
Using the server time zone, specify the earliest date/time to check CBR-index entries.
Can not be used with the delete option1.
u[e]:
P8 administrator username and password.
If the password is not specified then the user is prompted for it.
The PWCrypt encrypted password can be used with ue.
If the option is not used then P8 integrated logon is used.
reindex:
Re-index items that are missing from the CBR collections by updating the database or
1
Content or properties may be created before a specified date but indexed after. The application would flag the
CBR-indexed items for deletion.
creating new CE index requests.
delete:
Used to remove items from the CBR collections that should not be in the collections
(duplicates or deleted CE items).
Bulk insert files (BIF) are created that can be used to remove these items from the
collections and optionally the application can create new CE index requests to delete
the items from the CBR collections. If the application creates CE index requests (i.e. bc
not specified) the BIF files are created as a log.
bc:
Backward-compatible mode for re-index or delete.
The application can not create CE index requests prior to P8CE-3.5.2-016 so the
backward -compatible mode can be used to re-index or delete.
In this mode the application will re-index database content and property values by
directly updating the CE database2 and will create BIF files that must later be applied to
the collections to delete items.
File Store content can not be re-indexed in backward-compatible mode.
debug:
This option will not be documented. It is used to log every object id and element
sequence number being checked from the CE database and the CBR collections. The
data will be logged via the CE tracing with the 'Activity' flag.
Omitting the reindex and delete options will run the application in report-only mode. No modifications
will be made to the database or the CBR collections.
Logging
The output to the command line will report the application's progress. Here is a sample output:
C:\Program Files\FileNet\Content Engine>CEIxCheck.exe -check p018-1 -props -content -datapath c:\temp -reindex -delete
P8 CE 3.5.x CBR Index Check v1.1 - Check and fix CE 3.5.x CBR indexes
Copyright (c) 2009 IBM Corporation
1. Query for Id values of objects with content that should be indexed.
Run query:
SELECT object_id, content_info, content_info_2 FROM DocVersion WHERE storage_location = 'FNDB:/' AND content_info IS
NOT NULL AND home_id IS NULL ORDER BY object_id
Query completed.
2. Write the query results to files, split by the collection that should have the indexed content.
Total recordset results written: 23733
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-01-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-02-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-03-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-04-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-05-0000.dat
2
The application will update the pi_sequence_no or the ci_sequence_no on the Document, Annotation or Content
tables as appropriate. CE will detect updated rows as items that need indexing.
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-06-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-07-0000.dat
3. Query for Id values of objects with property values that should be indexed.
Run query:
SELECT object_id FROM DocVersion WHERE (u2e_documenttitle IS NOT NULL) AND home_id IS NULL ORDER BY object_id
Query completed.
4. Write the query results to files, split by the collection that should have the indexed property values.
Total recordset results written: 23733
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-01-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-02-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-03-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-04-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-05-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-06-0000.dat
Save recordset file c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-07-0000.dat
5. List all entries in one collection.
Open collection: \\BABEL\_FS\FS_p018-1\index\db0
Index Query:
Results: 6158
6. Sort by object Id.
Waiting for sort to complete.
7. Match the collection's entries with results from the database query for indexed properties. Create a reindex request for any
items not in the collection and mark items in the collection that are not in the database query results.
Check property results for collection 0
Open data file: c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
Open data file: c:\temp\CEIxCheck-Annot-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
File not found
Total recordset results read: 2912
Open data file: c:\temp\CEIxCheck-Doc-props-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0001.dat
File not found
8. Match the collection's entries with results from the database query for indexed content. Create a reindex request for any
items not in the collection and mark items in the collection that are not in the database query results.
Check content results for collection 0
Open data file: c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
Open data file: c:\temp\CEIxCheck-Annot-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0000.dat
File not found
Total recordset results read: 2912
Open data file: c:\temp\CEIxCheck-Doc-DB-{62D49B18-E136-401F-B837-99F1291CEF4F}-00-0001.dat
File not found
9. Query for marked items and delete these collection entries or write entries to a Verity BIF file. BIFs are processed by the user
after the application completes.
Delete marked index items
Index Query: $delete = 1
Results: 334
10. Repeat steps 5 to 9 for each collection.
Open collection: \\BABEL\_FS\FS_p018-1\index\db1
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db2
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db3
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db4
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db5
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db6
...
Open collection: \\BABEL\_FS\FS_p018-1\index\db7
...
Done.
Other logging is provided via the CEIndexCheck component in the CE trace logging. CE trace Logging can
enabled using EM, or TraceControl.exe. The CEIndexCheck logging component will be created and
enabled the first time the application is run.
The trace flags are:
 Activity: Generates a report of items that need to be re-indexed or deleted from the indexes.
 Error Conditions: Leave this on.
 General Info: Progress messages – same as information output to the command line.
 Procedure Entry or Exit: For diagnostic purposes only.
 Timer: Not used.