Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
&RS\ULJKW 8QGHUWKHFRS\ULJKWODZVQHLWKHUWKHGRFXPHQWDWLRQQRUWKHVRIWZDUHPD\EHFRSLHGSKRWRFRSLHG UHSURGXFHGWUDQVODWHGRUUHGXFHGWRDQ\HOHFWURQLFPHGLXPRUPDFKLQHUHDGDEOHIRUPLQZKROHRULQ SDUWZLWKRXWWKHSULRUZULWWHQFRQVHQWRI,%0&RUSRUDWLRQH[FHSWLQWKHPDQQHUGHVFULEHGLQWKH GRFXPHQWDWLRQRUWKHDSSOLFDEOHOLFHQVLQJDJUHHPHQWJRYHUQLQJWKHXVHRIWKHVRIWZDUH &RS\ULJKW²/RWXV'HYHORSPHQW&RUSRUDWLRQ &RS\ULJKW,%0&RUSRUDWLRQ /RWXV6RIWZDUH,%06RIWZDUH*URXS 2QH5RJHUV6WUHHW &DPEULGJH0$ $OO5LJKWV5HVHUYHG /LVWRI7UDGHPDUNV 'RPLQRFF0DLO1RWHV1RWHV%HQFK'RPLQR&RQQHFWRUV'RPLQR(QWHUSULVH&RQQHFWLRQ6HUYLFHV ,%0/RWXV(QWHUSULVH,QWHJUDWRUIRU'RPLQR/(,DQG1RWHV);DUHWUDGHPDUNVDQG)UHHODQFH )UHHODQFH*UDSKLFV/RWXV/RWXV1RWHV/RWXV6FULSW1RWHV0DLO1RWHV64/1RWHV9LHZ 2UJDQL]HU6PDUW,FRQVDQG6PDUW6XLWHDUHUHJLVWHUHGWUDGHPDUNVRI/RWXV'HYHORSPHQW&RUSRUDWLRQ DQGRU,%0&RUSRUDWLRQLQWKH8QLWHG6WDWHVRWKHUFRXQWULHVRUERWK26:DUSDQG3RZHU3&DUH WUDGHPDUNVDQG,%0$,;'%26L6HULHV]6HULHVDQGS6HULHV263UHVHQWDWLRQ0DQDJHUDQG 61$DUHUHJLVWHUHGWUDGHPDUNVDQG'%'%DQG56DUHWUDGHPDUNVRI,QWHUQDWLRQDO %XVLQHVV0DFKLQHV&RUSRUDWLRQ7LYROL&RXULHULVDWUDGHPDUNRI7LYROL6\VWHPVDQGRU,%0 &RUSRUDWLRQLQWKH8QLWHG6WDWHVRWKHUFRXQWULHVRUERWK $OORWKHUWUDGHPDUNVDUHWKHSURSHUW\RIWKHLUUHVSHFWLYHRZQHUV 'LVFODLPHU 7+,6'2&80(17$7,21,63529,'(')255()(5(1&(385326(621/<:+,/(())2576 :(5(0$'(729(5,)<7+(&203/(7(1(66$1'$&&85$&<2)7+(,1)250$7,21 &217$,1(',17+,6'2&80(17$7,217+,6'2&80(17$7,21,63529,'('´$6,6µ :,7+287$1<:$55$17<:+$762(9(5$1'727+(0$;,080(;7(173(50,77(' /2786',6&/$,06$//,03/,(':$55$17,(6,1&/8',1*:,7+287/,0,7$7,217+( ,03/,(':$55$17,(62)0(5&+$17$%,/,7<121,1)5,1*(0(17$1'),71(66)25$ 3$57,&8/$5385326(:,7+5(63(&7727+(6$0(/27866+$//127%(5(63216,%/( )25$1<'$0$*(6,1&/8',1*:,7+287/,0,7$7,21',5(&7,1',5(&7 &216(48(17,$/25,1&,'(17$/'$0$*(6$5,6,1*2872)7+(86(2)2527+(5:,6( 5(/$7('727+,6'2&80(17$7,2125$1<27+(5'2&80(17$7,21 127:,7+67$1',1*$1<7+,1*727+(&2175$5<127+,1*&217$,1(',17+,6 '2&80(17$7,2125$1<27+(5'2&80(17$7,21,6,17(1'('721256+$//+$9( 7+(())(&72)&5($7,1*$1<:$55$17,(6255(35(6(17$7,216)520/278625,76 6833/,(5625/,&(1625625$/7(5,1*7+(7(506$1'&21',7,2162)7+( $33/,&$%/(/,&(16($*5((0(17*29(51,1*7+(86(2)7+,662)7:$5( Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi LEI on the iSeries . . . . . . . . . . . . . . . . . . . . . . 16 Related Documentation . . . . . . . . . . . . . . . xi LEI Server on iSeries . . . . . . . . . . . . . . . . . 16 . . . . . . . xii LEI Server Running Window on iSeries . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Contact and Support Information 1 Introduction . . . . . . . . . . . . . . . . . . . . . . 1 Organization of this Manual . . . . . . . . . . . . . . . 1 Getting Started with LEI . . . . . . . . . . . . . . . . . . 3 Introduction to LEI . . . . . . . . . . . . . . . . . . . . . . 3 LEI Architecture ..................... 3 Lotus Connectors . . . . . . . . . . . . . . . . . . . . . 6 LEI Activities . . . . . . . . . . . . . . . . . . . . . . . . 7 LEI Logs ........................... 8 Obtaining Help and Opening User Documentation . . . . . . . . . . . . . . . . . . . . 9 User Assistant Help . . . . . . . . . . . . . . . . . . . 9 Pop-up Help . . . . . . . . . . . . . . . . . . . . . . . . 9 Field-level Help . . . . . . . . . . . . . . . . . . . . . . 9 User Documentation . . . . . . . . . . . . . . . . . . 9 General LEI Terms . . . . . . . . . . . . . . . . . . . . . 10 Languages Support on iSeries . . . . . . . . . . 17 Term Comparison for iSeries . . . . . . . . . . . 17 2 Starting and Running the LEI Server . . . . . . . . . . . . . . . . . . . . . . . . . 19 Starting the LEI Server . . . . . . . . . . . . . . . . . . 19 Executing LEI Server Commands From the Console . . . . . . . . . . . . . . . . . . . . . . 20 Executing LEI Server Commands from the Administrator . . . . . . . . . . . . . . . . . 20 Running LEI as a Domino Addin Task . . . . . . 21 Considerations . . . . . . . . . . . . . . . . . . . . . 21 Understanding the leiclean Script for UNIX and iSeries . . . . . . . . . . . . . . . . . 22 Starting and Running the LEI Server on iSeries . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Starting the LEI Server Automatically . . . . . . . . . . . . . . . . . . . . 23 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Starting the LEI Server Manually . . . . . . . . 23 Sample LEI Configuration with Processing Times . . . . . . . . . . . . . . . . . 13 Stopping the LEI Server on iSeries . . . . . . . . . 23 General Performance Considerations . . . . . 14 Advanced RealTime Activity: Indexing Increases Performance . . . . . . 15 Using Advanced RealTime Activities in a Clustered Domino Server Environment . . . . . . . . . . . . . . . . . . . . . 15 Web-based Application Considerations . . . . . . . . . . . . . . . . . . . 16 From the Domino Console . . . . . . . . . . . . . 23 From the LEI Administrator . . . . . . . . . . . 24 By stopping the Domino Server . . . . . . . . . 24 Determining if LEI Server is Running on iSeries . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Show Task from Domino Console . . . . . . . 24 WRKACTJOB on iSeries . . . . . . . . . . . . . . 25 From LEI Administrator Database . . . . . . . 25 iii 3 LEI Administrator . . . . . . . . . . . . . . . 27 . . . . . . . . . . . . . . . . . . . . . 27 5 Introduction to LEI Activities . . . . . . . . . . . . . . . . . . . . . . 57 LEI Administrator Database . . . . . . . . . . . . . . 28 Introduction to Activity Documents . . . . . . . . 57 . . . . . 28 Creating Activities . . . . . . . . . . . . . . . . . . . . . 58 LEI Administrator Action Bar . . . . . . . . . . . . . 29 Browsing Connections from within an Activity . . . . . . . . . . . . . . . . . . . . . . 58 LEI Administrator Configuring Administrator Security LEI Administrator Navigator and View Options . . . . . . . . . . . . . . . . . . . . . . . . . 31 Common Activity Fields and Buttons . . . . . . . 59 LEI Administrator Menu Commands . . . . . . . 38 Author Privileges Action Button . . . . . . . . 59 Create . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Activity Name and Status General Options . . . . . . . . . . . . . 59 . . . . . . . . . . . . . . . . . . . . 60 Creating, Editing, and Deleting Documents . . . . . . . . . . . . . . . . . . . . . . 40 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . 61 Creating New LEI Documents . . . . . . . . . . 40 Activity Scheduling Options . . . . . . . . . . . . . . 62 Editing Existing Documents . . . . . . . . . . . 42 Category and Comments . . . . . . . . . . . . . . 62 Overview of Scheduling . . . . . . . . . . . . . . 62 . . . . . . . . . . . . . . . . . 43 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . 65 Activity Document Action Bar . . . . . . . . . . . . 43 Advanced RealTime Activities . . . . . . . . . . . . 69 . . . . . . . . . . . . . . . . . . . 44 Virtual Fields Activity . . . . . . . . . . . . . . . . 69 . . . . . . . . . . . . . . . . . . . . . 44 Virtual Documents Activity . . . . . . . . . . . . 70 Start! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Virtual Agents Activity . . . . . . . . . . . . . . . 70 Save and Exit . . . . . . . . . . . . . . . . . . . . . . . 45 Integrated Credentials Database Deleting Documents Author Privileges Edit Document . . . . . . . . 71 . . . . . . . . . . . 45 Command Line Execution of Activities . . . . . . 75 Configuring Browsing . . . . . . . . . . . . . . . . 47 General Considerations . . . . . . . . . . . . . . . 75 LEI Configuration Documents . . . . . . . . . . . . 47 Additional iSeries Considerations . . . . . . . 76 Understanding Basic Browsing Administrator Configuration . . . . . . . . . . . 47 Data Mapping . . . . . . . . . . . . . . . . . . . . . . . . 76 Server Configuration . . . . . . . . . . . . . . . . . 48 According to Field Names . . . . . . . . . . . . . 77 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 According to Field Position . . . . . . . . . . . . 77 Server Log . . . . . . . . . . . . . . . . . . . . . . . . . 50 According to User Definition . . . . . . . . . . . 77 Activity Log . . . . . . . . . . . . . . . . . . . . . . . . 51 When Field Numbers Don’t Match LEI Log Operation Log . . . . . . . . . . . . . . . . . . . . . . 52 LEI Script Vault . . . . . . . . . . . . . . . . . . . . . . . 53 4 Introduction to Connectors . . . . . . 55 Lotus Connectors Supplied with LEI . . . . . . . 55 . . . . . . 77 Stored Procedures and Output Parameters . . . . . . . . . . . . . . . . . . . . . . 78 Example: Notes to DB2 Direct Transfer Activity . . . . . . . . . . . . . . . . . . 79 Moving Text List Data over Different Platforms . . . . . . . . . . . . . . . . . . . . . . . 81 Activity Logging . . . . . . . . . . . . . . . . . . . . . . 82 iv IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 6 Admin Backup Activity . . . . . . . . . . 85 9 Command Activity . . . . . . . . . . . . . 101 Introduction to the Admin Backup Activity . . . . . . . . . . . . . . . . . . . . . . . . . 85 Introduction to the Command Activity . . . . . 101 When to Use the Admin Backup Activity . . . . . . . . . . . . . . . . . . . . . . . . . 85 How to Create a Command Activity . . . . . . . 101 How to Create an Admin Backup Activity . . . . . . . . . . . . . . . . . . . . . . . . . 85 Identification . . . . . . . . . . . . . . . . . . . . . . 103 Admin Backup Activity Document . . . . . . . . . 86 When to Use a Command Activity . . . . . . . . 101 Command Activity Document . . . . . . . . . . . 102 Connection . . . . . . . . . . . . . . . . . . . . . . . 103 Identification . . . . . . . . . . . . . . . . . . . . . . . 87 Command . . . . . . . . . . . . . . . . . . . . . . . . 103 Backup Options . . . . . . . . . . . . . . . . . . . . . 87 Command Activity Examples . . . . . . . . . . . . 104 . . . . . . . . 104 7 Admin Purge Log Activity . . . . . . . 89 Command Activity Using SQL Introduction to the Admin Purge Log Activity . . . . . . . . . . . . . . . . . . . . . . . . . 89 Command Activity Using a Notes Agent . . . . . . . . . . . . . . . . . . . . . . . . . 105 When to Use the Admin Purge Log Activity . . . . . . . . . . . . . . . . . . . . . . . . . 89 Command Example Issuing Operating System Commands . . . . . . . 107 How to Create an Admin Purge Log Activity . . . . . . . . . . . . . . . . . . . . . . . . . 89 10 Direct Transfer Activity . . . . . . . 109 Saving Selected Logs before Purging . . . . . 89 Admin Purge Log Activity Document . . . . . . 90 Identification . . . . . . . . . . . . . . . . . . . . . . . 90 Purge Log Options . . . . . . . . . . . . . . . . . . 91 8 Archive Activity . . . . . . . . . . . . . . . . . 93 Introduction to the Archive Activity . . . . . . . . 93 When to Use the Archive Activity . . . . . . . . . 93 Combining an Archive Activity with other Activities . . . . . . . . . . . . . . . . . . . 94 Introduction to the Direct Transfer Activity . . . . . . . . . . . . . . . . . . . . . . . . 109 When to Use a Direct Transfer Activity . . . . 109 How to Create a Direct Transfer Activity . . . . . . . . . . . . . . . . . . . . . . . . 109 Direct Transfer Activity Document . . . . . . . . 110 Identification . . . . . . . . . . . . . . . . . . . . . . 110 Source . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Target . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 112 How to Create an Archive Activity . . . . . . . . . 94 Direct Transfer Options . . . . . . . . . . . . . . 113 Archive Activity Document . . . . . . . . . . . . . . 95 Precision Options . . . . . . . . . . . . . . . . . . 113 Identification . . . . . . . . . . . . . . . . . . . . . . . 95 Source Data Options . . . . . . . . . . . . . . . . 114 Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Target Data Options Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Performance Options . . . . . . . . . . . . . . . . 117 Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Direct Transfer Activity Considerations . . . . 118 Archive Selection Options . . . . . . . . . . . . . 98 Using Stored Procedures and Output Parameters . . . . . . . . . . . . . . . . . . . . . 118 Archive Precision Options . . . . . . . . . . . . . 99 Archive Target Data Options . . . . . . . . . . 100 . . . . . . . . . . . . . . . . 115 Using the Lotus Connector for Notes . . . . 118 Contents v Using the Lotus Connector for File System . . . . . . . . . . . . . . . . . . . . . . . . 119 Step 12: Process the Virtual Fields Activity . . . . . . . . . . . . . . . . . . . . . . . . 147 Using Oracle or ODBC to Create Target Metadata . . . . . . . . . . . . . . . . . 119 Creating a Virtual Fields Activity without the User Assistant . . . . . . . . . 147 11 Polling Activity . . . . . . . . . . . . . . . 121 Step 1: Open the Activity Document . . . . 147 . . . . . . . 121 Step 2: Enter a Name for the Activity . . . . 147 When to Use the Polling Activity . . . . . . . . . 121 Step 3: Select the Domino Application and the Form to Monitor . . . . . . . . . . . . . . . . . . . . . . . . 147 Introduction to the Polling Activity How to Create a Polling Activity . . . . . . . . . 122 Polling Activity Document . . . . . . . . . . . . . . 123 Identification . . . . . . . . . . . . . . . . . . . . . . 124 Step 4: Select the Lotus Connection for the Data Source . . . . . . . . . . . . . . . 148 . . . . . . . . . . . . . 148 Connection . . . . . . . . . . . . . . . . . . . . . . . 124 Definition . . . . . . . . . . . . . . . . . . . . . . . . 126 Step 6: Mapping Key and Data Fields . . . . 149 Error Handing Options . . . . . . . . . . . . . . 128 Step 7: Select Event(s) to Monitor . . . . . . . 150 Considerations for Polling Activity Scheduling . . . . . . . . . . . . . . . . . . . . . 128 Step 8: Select Options and Event Options . . . . . . . . . . . . . . . . . . . . . . . . 150 Step 5: Enable Subforms . . . . . . 150 12 Virtual Fields Activity . . . . . . . . . 131 Step 9: Select Scheduling Options Introduction to Virtual Fields Activities . . . . 131 Step 10: Save, Initialize Keys, and Close the Document . . . . . . . . . . . . . . 150 Usage Requirements, Performance, and Mapping in Virtual Fields Activities . . . . . . . . . . . . . . . . . . . . . . . 134 Creating a Virtual Fields Activity Using the User Assistant . . . . . . . . . . . . . . . . 136 Step 1: Open the Activity Document . . . . 136 Step 11: Process the Virtual Fields Activity . . . . . . . . . . . . . . . . . . . . . . . . 151 Virtual Fields Activity Document Options . . . . . . . . . . . . . . . . . . . . . . . . 152 General Options in Virtual Fields . . . . . . 153 Step 2: Select the Domino Database . . . . . 137 Initialize Keys in Virtual Fields . . . . . . . . 157 Step 3: Select the Notes Form to Monitor . . . . . . . . . . . . . . . . . . . . . . . . 138 Multi-value Data in Virtual Fields . . . . . . 158 Step 4: Select the Lotus Connection for the Data Source . . . . . . . . . . . . . . . 139 Integrated Credentials in Virtual Fields . . . . . . . . . . . . . . . . . . . . . . . . . 161 Virtual Attachments . . . . . . . . . . . . . . . . 163 Step 5: Map Key and Data Field(s) . . . . . . 141 Creating a Virtual Attachments Table . . . . 167 Step 6: Select Events to Monitor . . . . . . . . 142 Event Options Create, Open, Update, and Delete . . . . . . . . . . . . . . . . . . . . . . 169 Step 7: Name the Activity Document . . . . 143 Create Options . . . . . . . . . . . . . . . . . . . . 170 Step 9: Select Options . . . . . . . . . . . . . . . . 145 Open Options . . . . . . . . . . . . . . . . . . . . . 172 Step 10: Select Scheduling Options . . . . . . 146 Update Options . . . . . . . . . . . . . . . . . . . . 174 Step 11: Save, Initialize Keys, and Close the Document . . . . . . . . . . . . . . 146 Delete Options . . . . . . . . . . . . . . . . . . . . . 176 Step 8: Enable Subforms . . . . . . . . . . . . . 144 vi IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Running Virtual Agents Against Virtual Fields . . . . . . . . . . . . . . . . . . . . . . . . . 178 Step 10: Save and Close the Document . . . . . . . . . . . . . . . . . . . . . . 202 13 Virtual Documents Activity . . . . 179 Step 11: Process the Virtual Documents Activity . . . . . . . . . . . . . . 202 Introduction to Virtual Documents Activities . . . . . . . . . . . . . . . . . . . . . . . 179 Comparing Virtual Documents and Virtual Fields . . . . . . . . . . . . . . . . . . . 179 Virtual Documents and Monitored Forms . . . . . . . . . . . . . . . . . . . . . . . . . 183 Virtualizing Preexisting Data . . . . . . . . . . 184 Required External System Metadata for Virtual Documents Activities . . . . . . . 185 Virtual Documents Required Column Data Types . . . . . . . . . . . . . . 187 Creating a Virtual Documents Activity without the User Assistant . . . . . . . . . 203 Step 1: Open the Activity Document . . . . 203 Step 2: Enter a Name for the Activity . . . . 203 Step 3: Select the Domino Application and the Form to Monitor . . . . . . . . . . . . . . . . . . . . . . . . 203 Step 4: Select the Lotus Connection for the Data Source . . . . . . . . . . . . . . . 204 Step 5: Mapping . . . . . . . . . . . . . . . . . . . 204 Using and Testing the Required External System Fields . . . . . . . . . . . . 189 Step 6: Create the External Key Table (optional) . . . . . . . . . . . . . . . . . . . . . . 204 Indexing to Increase Performance . . . . . . . . . 191 Step 7: Select Options and Event Options . . . . . . . . . . . . . . . . . . . . . . . . 204 Special Considerations When Using Public Key Encrypted Fields . . . . . . . . 192 Special Considerations When Deleting Virtual Documents . . . . . . . . . . . . . . . 193 Deletion Stubs . . . . . . . . . . . . . . . . . . . . . 193 Soft Deletions . . . . . . . . . . . . . . . . . . . . . 194 Creating a Virtual Documents Activity Using the User Assistant . . . . . . . . . . . 194 Step 1: Open the Activity Document . . . . 195 Step 8: Select Scheduling Options . . . . . . 204 Step 9: Save and Close the Document . . . . 204 Step 10: Process the Virtual Documents Activity . . . . . . . . . . . . . . 205 Virtual Documents Activity Document Options . . . . . . . . . . . . . . . . . . . . . . . . 205 Virtual Documents Options Tab . . . . . . . . . . 206 General Options . . . . . . . . . . . . . . . . . . . 206 Step 2: Select the Domino Database . . . . . 196 Creating an External Key Table . . . . . . . . 209 Step 3: Select the Notes Form to Monitor . . . . . . . . . . . . . . . . . . . . . . . . 197 Integrated Credentials . . . . . . . . . . . . . . . 215 Step 4: Select the Lotus Connection for the Data Source . . . . . . . . . . . . . . . 198 Creating a Virtual Attachments Table . . . . 220 Step 5: Map Data Field(s) . . . . . . . . . . . . . 200 Step 6: Create the External Key Table (optional) . . . . . . . . . . . . . . . . . . . . . . 200 Virtual Attachments . . . . . . . . . . . . . . . . 217 Event Options Tab . . . . . . . . . . . . . . . . . . . . 223 Create Options . . . . . . . . . . . . . . . . . . . . 223 Open Options . . . . . . . . . . . . . . . . . . . . . 225 Step 7: Name the Activity Document . . . . 201 Update Options . . . . . . . . . . . . . . . . . . . . 226 Step 8: Select Options . . . . . . . . . . . . . . . . 202 Delete Options . . . . . . . . . . . . . . . . . . . . . 228 . . . . . . 202 Using Stored Procedures in Virtual Documents Events . . . . . . . . . . . . . . . 229 Step 9: Select Scheduling Options Contents vii Stored Procedure Examples . . . . . . . . . . . 231 General Options in Virtual Agents . . . . . . 246 Running Virtual Agents Against Virtual Documents . . . . . . . . . . . . . . . . . . . . . 235 Integrated Credentials in Virtual Agents . . . . . . . . . . . . . . . . . . . . . . . . 248 14 Virtual Agents Activity . . . . . . . . 237 Executing Virtual Agents from HTTP Client . . . . . . . . . . . . . . . . . . . . . . . . . 249 Introduction to Virtual Agents Activities . . . . 237 Stored Procedure Parameters . . . . . . . . . . 238 Running Virtual Agents Against Virtual Documents . . . . . . . . . . . . . . . . . . . . . 250 Creating a Virtual Agents Activity Using the User Assistant . . . . . . . . . . . 238 Running Virtual Agents Against Virtual Fields . . . . . . . . . . . . . . . . . . . . . . . . . 250 Step 1: Open the Activity Document . . . . 239 Step 2: Select the Domino Database to Monitor . . . . . . . . . . . . . . . . . . . . . . 240 Step 3: Select the Lotus Connection for the External Data Source . . . . . . . . 241 Step 4: Select the Owner . . . . . . . . . . . . . . 242 Step 5: Name the Activity Document . . . . 242 Step 6: Select Options and Event Options . . . . . . . . . . . . . . . . . . . . . . . . 243 Step 7: Select Scheduling Options . . . . . . 243 Step 8: Save and Close the Document . . . . 243 Step 9: Process the Virtual Agents Activity . . . . . . . . . . . . . . . . . . . . . . . . 244 Creating a Virtual Agents Activity without the User Assistant . . . . . . . . . 244 Step 1: Open the Activity Document . . . . 244 Step 2: Enter a Name for the Activity . . . . 244 Step 3: Select the Domino Application to Monitor . . . . . . . . . . . . 244 Step 4: Select the Lotus Connection for the Data Source . . . . . . . . . . . . . . . 245 Step 5: Select Options and Event Options . . . . . . . . . . . . . . . . . . . . . . . . 245 Step 6: Select Scheduling Options . . . . . . 245 Step 7: Save and Close the Document . . . . 245 Step 8: Process the Virtual Agents Activity . . . . . . . . . . . . . . . . . . . . . . . . 245 Virtual Agents Activity Document Options . . . . . . . . . . . . . . . . . . . . . . . . 246 15 Replication Activity . . . . . . . . . . . 251 Introduction to the Replication Activity . . . . 251 When to Use the Replication Activity . . . . 251 Types of Replication Activities . . . . . . . . . 252 Using Database Triggers with Timestamp Replication . . . . . . . . . . . . 256 Specifying a Selection View in a Notes Database . . . . . . . . . . . . . . . . . . 256 Sort Order Considerations . . . . . . . . . . . . 256 How to Create a Replication Activity . . . . . . 257 Replication Activity Document . . . . . . . . . . . 258 Special Settings . . . . . . . . . . . . . . . . . . . . 259 Identification . . . . . . . . . . . . . . . . . . . . . . 259 Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 261 Replication Options . . . . . . . . . . . . . . . . . 263 Using the Do Not Replicate Timestamp Option . . . . . . . . . . . . . . . . . . . . . . . . 270 Using Restrictions in Keyed Replication Activities . . . . . . . . . . . . . 273 Using Restrictions in Timestamp Replication . . . . . . . . . . . . . . . . . . . . . 274 Replication Examples . . . . . . . . . . . . . . . . . . 276 Skip Insertions . . . . . . . . . . . . . . . . . . . . . 276 Skip Updates . . . . . . . . . . . . . . . . . . . . . . 277 Skip Deletions . . . . . . . . . . . . . . . . . . . . . 278 16 Java Activity . . . . . . . . . . . . . . . . . . 279 Introduction to the Java Activity . . . . . . . . . . 279 viii IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide When to Use the Java Activity . . . . . . . . . 279 How to Create a Java Activity . . . . . . . . . . . 279 Java Activity Document . . . . . . . . . . . . . . . . 280 Identification . . . . . . . . . . . . . . . . . . . . . . 281 Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 17 Scripted Activity . . . . . . . . . . . . . . 283 Introduction to the Scripted Activity . . . . . . 283 How LEI Supports LotusScript . . . . . . . . . . . 285 Running Unrestricted LotusScript Agents . . . . . . . . . . . . . . . . . . . . . . . . 285 How to Create a Scripted Activity . . . . . . . . 286 Scripted Activity Document . . . . . . . . . . . . . 286 Identification . . . . . . . . . . . . . . . . . . . . . . 287 Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Agent Development Document . . . . . . . . 288 Determining if an Agent is Running Scripted Activity Considerations . . . . 290 . . . . . . . . . 291 Appendix A LEI and Data . . . . . . . . 293 Appendix B Error Messages and Troubleshooting . . . . . . . . . . . . . 297 Appendix C Tutorials . . . . . . . . . . . . 321 Appendix D Using Virtual Fields with Virtual Documents . . . . 377 Appendix E Using Virtual Activities in a Domino Cluster . . . . 393 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 Contents ix Preface This document provides information about IBM Lotus Enterprise Integrator for Domino (LEI). It describes how to use the LEI server and LEI Administrator. It also describes each LEI Activity. Tutorials are available in the last appendix. Information about Lotus Connectors and connectivity is described in the companion document entitled Lotus Connectors and Connectivity Guide (lccon6.nsf). This documentation file is installed with Domino, not with LEI. The lccon6.pdf file is available for download at http://www.notes.net. Related Documentation This section lists documentation that you may find useful as you learn about and use LEI. Note Two of the four LEI user documentation files (Lotus Connectors and Connectivity Guide - lccon6.nsf and Lotus Connector LotusScript Extensions Guide - lsxlc6.nsf) are also part of the DECS documentation set and are now supplied with Domino only, not LEI. To obtain these two needed documentation databases, you must either install DECS or perform a custom install of the Domino server and select the DECS User Assistance option. These two files are also available at http://www.notes.net. Their PDF equivalents are available only at http://www.notes.net. LEI Documentation Set Documentation for LEI is available online with the product distribution and at http://www.notes.net. The LEI documentation set is listed below: • IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide — Provides information and instructions for using LEI and its Activities (leidoc.nsf and leidoc.pdf). • IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide — Provides installation, configuration, and migration instructions for LEI (leiig.nsf and leiig.pdf). • Lotus Connectors and Connectivity Guide — Provides information on how to set up and use the supplied Lotus Connectors, including information about required software and instructions for testing connectivity (lccon6.nsf). This documentation file is installed with Domino, not with LEI. The lccon6.pdf file is available for download at http://www.notes.net. xi • Lotus Connector LotusScript Extensions Guide — Describes the Lotus Connector LotusScript Extensions, which can be used in writing scripted sessions for accessing enterprise data (lsxlc6.nsf). This documentation file is installed with Domino, not with LEI. The lsxlc6.pdf file is available for download at http://www.notes.net. • IBM Lotus Enterprise Integrator for Domino (LEI) Release Notes — The release notes (readme.txt) contain information about the current release of LEI that may not be included in the documentation set. Current information about LEI can also be found at http://www.lotus.com/ei. Other Documentation For more information about related tools, refer to the following documents: • Domino Administrator’s Guide — This manual provides information for configuring and administering a Notes installation. • LotusScript Language Reference — This manual provides information about writing LotusScript programs. This could be useful if you want to use the Domino LotusScript Extensions (LSX) to write custom Activities. • Additional Lotus Connector Documentation — Lotus Software, IBM Software Group sells additional Lotus Connectors for enterprise systems including Enterprise Resource Planning (ERP) and Transaction Processing Systems. Specific documentation about those Connectors is included with the specific Connector software and package. You may need documentation for the specific databases, ERP, or transaction processing systems that you are using. Contact and Support Information Lotus Software, IBM Software Group provides extensive support for its products. Enterprise Integrator Web Site To obtain the latest information about LEI and DECS, visit the following Web site: http://www.lotus.com/ei Note Domino-related product updates for iSeries can be found at following Web site: http://www-1.ibm.com/servers/eserver/iseries/domino/support/qm u.htm Technical Support To contact Customer Support with suggestions or questions regarding your Lotus Connector or LEI/DECS application, call 1-800-346-6388. xii IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 1 Introduction This chapter provides information about the organization of this manual and terms you should know when working with IBM Lotus Enterprise Integrator for Domino (LEI). This chapter also includes an overview of LEI, including a description of its features, functionality, and architecture. Organization of this Manual This manual includes the sections described in the following table. Section Description Chapter 1 Introduction This chapter provides information about the organization of this manual and terms you should know when working with LEI. This chapter also includes an introduction to LEI, with an overview of the architecture. It also states how to contact Lotus. Chapter 2 Starting and Running the LEI Server This chapter provides information about starting the LEI server and using the server commands. Also included are instructions for configuring the LEI server to automatically start at system startup. Chapter 3 LEI Administrator This chapter provides an introduction to the LEI Administrator and information on using it. Chapter 4 This chapter points to the companion manual Lotus Introduction to Connectors Connectors and Connectivity Guide. Chapter 5 Introduction to LEI Activities This chapter introduces LEI Activities. It includes descriptions of items that are common to all Activities, and provides basic information for working with Activity documents. Chapter 6 Admin Backup Activity This chapter provides information about the Admin Backup Activity. Chapter 7 Admin Purge Log Activity This chapter provides information about the Admin Purge Log Activity. Chapter 8 Archive Activity This chapter provides information about the Archive Activity. continued 1 Section Description Chapter 9 Command Activity This chapter provides information about the Command Activity. Chapter 10 Direct Transfer Activity This chapter provides information about the Direct Transfer Activity. Chapter 11 Polling Activity This chapter provides information about the Polling Activity. Chapter 12 Virtual Fields Activity This chapter provides information about the Virtual Fields Activity. This is one of three LEI Advanced RealTime Activities. This chapter provides information about the Virtual Chapter 13 Virtual Documents Activity Documents Activity. This is one of three LEI Advanced RealTime Activities. Chapter 14 Virtual Agents Activity This chapter provides information about the Virtual Agents Activity. This is one of three LEI Advanced RealTime Activities. Chapter 15 Replication Activity This chapter provides information about the Replication Activity. Chapter 16 Java Activity This chapter provides information about the Java Activity. Chapter 17 Scripted Activity This chapter provides information about the Scripted Activity. Appendix A LEI and Data This appendix contains information about how LEI maps data types between different applications. Appendix B Error Messages and Troubleshooting This appendix contains information about troubleshooting LEI. Appendix C Tutorials This appendix contains tutorials for creating LEI Activities. Appendix D Using Virtual Fields with Virtual Documents This appendix provides insight into using the Virtual Fields Activity in conjunction with the Virtual Documents Activity. Appendix E This appendix contains practical considerations when Using Virtual Activities in a using the LEI Advanced RealTime Activities in a Domino Cluster Domino cluster. Note The Character Sets appendix from the LEI Release 3.2 documentation has been moved to the Lotus Connector LotusScript Extensions Guide lsxlc6.nsf supplied with Domino and located also at http://www.notes.net. 2 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Getting Started with LEI The steps below summarize how to get started with LEI. 1. Install LEI. See the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide. For installation and configuration issues, see the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide. Also see “Introduction to LEI,” in this chapter, and also the Troubleshooting appendix in this document. If upgrading and migrating from DECS or LEI Release 3.n, also see the Migration chapter in the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide. 2. Establish and test database connectivity. See the Lotus Connectors and Connectivity Guide. 3. Start the LEI server. See the chapter entitled “Starting and Running the LEI Server.” 4. Build connections for each external data source. See the Lotus Connectors and Connectivity Guide. 5. Build activities. See the chapter that describes the type of activity you want to create. Also see the chapter entitled “Introduction to LEI Activities.” 6. Schedule and run activities. See the chapter that describes the type of activity you want to create. Also see the chapter entitled “Introduction to LEI Activities.” 7. In case of difficulties, see the Troubleshooting appendix. 8. For practice examples, see the Tutorials appendix. Introduction to LEI IBM Lotus Enterprise Integrator for Domino (LEI) is an enterprise integration tool that moves data between disparate external data sources. LEI Architecture LEI is a data distribution application designed to enable enterprise-wide data access across multiple platforms. It is comprised of two main components: • LEI Server • LEI Administrator Chapter 1: Introduction 3 Overview The figure below illustrates the components of the LEI architecture and shows how data is processed by LEI. The LEI Administrator is used to administer the configuration and operational elements of the LEI system, including Connectors, Metaconnectors, and Activities. The LEI server periodically polls the LEI Administrator database for Activities to execute. When it finds an Activity that is scheduled to run, it executes that Activity. An Activity defines the actions that the LEI server will perform, such as transfer of data, replication of data, file system commands and so on. Each component is described in more detail in the sections below. 4 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LEI Server The LEI server is an engine that polls the LEI Administrator database (also referred to as the Control Store) for instructions to execute. These instructions are in the form of LEI Activities. Activities can be declarative or scripted. Declarative Activities are formsbased Activities created with the LEI Administrator. Scripted Activities are written in LotusScript, using the Lotus Connector LSX with LEI LotusScript Extensions. Once an Activity is written, it can be run from the Administrator or from the operating system command line. The LEI server runs as a multithreaded, multiprocessing task on server operating systems. It performs the actual work in transferring data between external sources, such as between Oracle and Sybase or between SAP and Notes, etc. The LEI server is described in the chapter entitled “Starting and Running the LEI Server.” Note Client libraries of the external systems to be accessed must be installed on the LEI server and the Domino server. When running activities, if the LEI server that you want the activity to run on, and the Domino server where the LEI Administrator resides are on separate machines, the client libraries must be installed on both machines. LEI Administrator The LEI Administrator is a Notes application that enables users to create Connections and Activities. It uses a variety of forms and documents, including: • Configuration • Connections • Metaconnections • Activities See subsequent sections for more information about each of these items. The LEI Administrator is described in the chapter entitled “LEI Administrator.” Chapter 1: Introduction 5 Lotus Connectors Lotus Connectors provide connectivity to external data sources. Connection Documents specify the names and network locations of each source. Use the LEI Administrator form specific to the type of connection you want to build. LEI provides Lotus Connectors for the following data sources: • Notes • DB2 • File System • Open Database Connectivity (ODBC) • Oracle 7 • Oracle 8 • OLE DB • Sybase • Text Before building connections, verify connectivity using the communications software appropriate to the specific data source. For more information about establishing and testing connectivity, refer to the Lotus Connectors and Connectivity Guide. For iSeries users, check with your system administrator to determine what Lotus Connectors are available. Connection options enable you to modify operational aspects of specific connections. They appear as fields within the Connection Document. For information about building connections, refer to the Lotus Connectors and Connectivity Guide. A Metaconnector is a special LEI Connector. A metaconnection “wraps around” a base connection, appearing to the Activity like the base connection but providing additional functionality. LEI provides the following Metaconnectors: • Order — The Order Metaconnector can be used to obtain a consistent ordering of results. For example, dictating a common ordering scheme between EBCDIC and ASCII server data, resulting in accurate replication and data evaluation operations. The Order Metaconnection orders the result set based on user-defined criteria. 6 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • Collapse/Expand — The Collapse/Expand Metaconnector collapses multiple records with a common key value into a single record with multi-value fields, or expands multi-value fields into multiple records. It can be used to transfer multiple source records to a single target record, or vice versa. It collapses records on fetch operations, and expands them on write operations according to user-defined criteria. • Meter — The Meter Metaconnector measures data access rates and volumes and provides other statistical information. • Trace — The Trace Metaconnector enables you to capture trace data. Options exist for specifying a timestamp, where to capture trace data, and what file name to use for logging output. This Metaconnection is usually used in a customer support-assisted situation. Note Metaconnectors add an additional level of functionality. Metaconnector functionality is included in the Advanced RealTime Activities. For other LEI Activities, they are defined in a separate Connection Document. Metaconnectors are described fully in the companion document Lotus Connectors and Connectivity Guide. LEI Activities Activities are the core of LEI. An LEI Activity tells the LEI server what to do and when to do it. You define Activities using the LEI Administrator. Specific types of LEI Activities are listed below: • Admin Backup Activity — Provides a way to schedule and execute regular backups of your LEI Administrator databases. • Admin Purge Logs Activity — Provides a way to schedule and execute the purging of LEI logs. • Archive Activity — Provides a way to schedule and execute regular archiving of your databases. • Command Activity — Enables execution of operating system and database commands. • Direct Transfer Activity — Allows you to transfer data between a source and target database by specifying SQL queries, Notes formulas or other data source-specific commands. • Java Activity — Provides a way to schedule and execute Java programs with the LEI server. Chapter 1: Introduction 7 • Polling Activity — Allows you to monitor data sources for specified conditions. When a condition is met, LEI immediately executes one or more specified activities. • Advanced RealTime Activities — Enables a Notes form for real-time interactive access with external system data. The Advanced RealTime Activities are Virtual Fields, Virtual Documents, and Virtual Agents. • Replication Activity — Enables data synchronization between dissimilar databases. • Scripted Activity — In addition to the standard declarative Activities that can be defined using the LEI Administrator, LEI also provides LotusScript Extensions that can be used to create scripted activities. This enables you to develop more advanced Activities that can perform data manipulation and data massaging. For more information, see the Lotus Connector LotusScript Extensions Guide. Each LEI Activity is described in its own chapter in this manual. LEI Logs LEI maintains a system log database that tracks system performance. These logs are helpful in assessing the performance of specific Activities, and are useful for troubleshooting. LEI maintains three types of logs: • Activity Logs — These track individual Activities and provide performance statistics. • Operation Logs — These track any errors encountered when running an Activity. • Server Logs — These track the operation and performance of LEI servers. In any activity view, you can see the log display for the most recent run of that activity using the Log action button. LEI logs are described in more detail in the chapter entitled “LEI Administrator.” 8 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Obtaining Help and Opening User Documentation All Activity Documents and Connection Documents are supplied with pop-up help and field-level help incorporated into the user interface. Online manuals are also available in .nsf form from the LEI Administrator Navigator and in .pdf form as supplied with LEI and also at http://www.notes.net. User Assistant Help You can optionally enable the User Assistant to assist you in creating an Advanced RealTime Activity or in creating or editing a Connection Document. The User Assistant prompts you the through the creation of a new Advanced RealTime Activity. With the User Assistant enabled, new and existing Connection Documents open with explanatory text at the top of the document. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. Pop-up Help Every LEI Activity Document and Connection Document includes pop-up help for each of the sections and most of the fields. This help is designed to make the requirements for the associated field easier to understand. Dark blue text indicates that pop-up help is available. To display pop-up help for a particular user interface option, position your cursor on the option and press any mouse button. Pop-up help appears immediately. Field-level Help Field-level help is available for all editable fields. It is provided in the bottom area of the user interface. To display field-level help, you must be in Edit mode. Once you are in Edit mode, field-level help appears when you position the cursor on any field where user input is accepted. Field-level help describes action options for each field on the Activity Document or Connection Document. User Documentation The LEI documentation set is comprised of four manuals as described in the preface of this document. The manuals are supplied in .nsf and .pdf format. To open a particular manual in .nsf format, click the Help option on the LEI Administrator and select one of the resultant four title choices. Once the manual is open, use its table of contents of topic choices to quickly navigate to the needed information. LEI user documentation is also available at the Lotus Developer Domain (LDD) at http://www.notes.net. Chapter 1: Introduction 9 General LEI Terms Because LEI works with several database products, directories, ERP systems, and transaction processing systems, it’s important to understand the vocabulary used in LEI and in other data sources. Term Description Column Referred to as a field in LEI, a set of similar data such as phone numbers or customer names. Usually displayed in column format, such as in a Notes View. Database A collection of metadata and data. A Lotus Domino database is stored as a file with the extension .nsf. Other systems generally abstract the database from the file system. Document Referred to in LEI as a record. A Notes document is a database entry that contains data. It is the equivalent of a database row and appears in row format in a Notes View. Field In LEI, the metadata description or data for one element of a record. Each record contains one or more fields, each with a data type, data value and other properties. Form The standard metadata object in Notes. A form defines the fields and data type used to create and display documents in a Domino database. Index A collection of key values enabling rapid searches. In Domino, a View is an index. Metadata Metadata describes the types, names, and collections of data within a database. Parameter Parameters are used to pass data or other information to or from procedures. Procedure A packaged set of operations in an external system. Procedures (also called Stored Procedures or Remote Procedure Calls) are executed to produce a result set, modify external data, or perform some other action. Procedures obtain input data through parameters and return data either through output parameters or returned result sets. continued 10 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Term Description Record An element of data composed of one or more fields. Examples are a row in a relational database or a document in Notes. Result set The set of data (rows or records) resulting from a selection operation. For example, the data to be transferred after being selected through a Notes selection statement or a SQL query would be a result set. Row In a relational database, the equivalent of a record. Statement A statement in a system’s native language syntax. Statements can produce result sets (selection statements) or invoke modifications, commands, or procedures. Examples include a Notes formula or SQL query. Table Referred to in LEI as metadata. A set of data in a relational database stored as rows and columns. View In Domino, a View collates and displays a list of documents with selected fields and acts as an index to those documents. In relational databases, a view is a customized presentation of data from one or more tables. Accessibility Accessibility features help a user who has a physical disability, such as restricted mobility or limited vision, to use software products successfully. LEI adheres to the IBM and Lotus Notes Accessibility guidelines. See http://www-3.ibm.com/able/guidelines.htm for more information on Accessibility. LEI provides keyboard access as an alternative or to augment its graphical user interface. The main LEI Navigation screen is a series of frames on a frameset. To navigate from one frame to another, press the <F6> key to move down and right or Shift <F6> to move up and left. Action buttons may be accessed by pressing the <Alt> key while in a frame and pressing the number displayed on the button. Chapter 1: Introduction 11 Note As an alternative to the Add Connection and Add Activity action buttons on the LEI Administrator, you can select the Create button in the top bar to yield the following pulldown menu. Once you open a Connection Document or Activity Document, you can press the tab key or the <Ctrl>E keys to enter Edit mode. You can tab from one document field to another. You can also use the arrow keys to move the cursor and change focus. You can use the Return key to select an action, such as toggling a document option on or off. Note When you are in Edit mode, field level help displays automatically for whatever field has focus. Field help appears at the bottom of the user interface. Pop-up help is also available for all dark blue Connection Document and Activity Document options using the mouse, but pop-up help is not available from the keyboard. To simplify user interaction with the application, LEI supports the shortcut keys listed next to the menu options on the top bar, such as File, Edit, View, and so on. To use the keyboard to open a top bar menu and then display help on any of its content, press <Alt> and the underlined character (for example, F for File, E for Edit). Keyboard equivalents are listed to the right of all pulldown menu choices. You can use screen-reader software and a digital speech synthesizer to hear what is displayed on the screen. You can also use voice recognition software, such as IBM ViaVoice, to enter data and to navigate the user interface. 12 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Performance This section contains useful configuration tips for obtaining the best LEI performance. For performance considerations that are specific to LEI Advanced RealTime activities, see the Virtual Documents, Virtual Fields, and Virtual Agents chapters later in this guide. Sample LEI Configuration with Processing Times This section contains parameters and processing times for a sample Replication Activity and Direct Transfer Activity using the supplied sample database leiempsamp.nsf. Data transfer was done across a network. In this sample configuration, the following parameters were present: • Platform and OS = Windows NT SP6a • Domino Server running Domino R6 • LEI Release 6 Server • RAM total = 163,248K • RAM available = 56,560K • Hard drive = 4GB • Free Space = 3GB • pagefile.sys = 163,840K (160MB) Using this test scenario, the Replication Activity with “Log Conflicts On” enabled and a 10,000 record database with conflicts will complete in 1-5 minutes depending on your external system and connection choice. Using this test scenario, the Direct Transfer Activity and a 10,000 record database will complete in 0.5-3 minutes depending on your external system and connection choice. Chapter 1: Introduction 13 General Performance Considerations • Array Transfer Array transfer, also known as bulk transfer, is a term used to describe the ability of a Connector to transfer many records at once. The benefit of transferring many records at once is that only one network transaction is needed, rather than many, for a given set of records. Using array transfer, the data has the same number of bytes but the transfer takes less time. Array transfers can be used for read (fetch) or write (insert) operations, depending on the RDBMS and the Connector. The Performance section of the Direct Transfer Activity document contains options that pertain to array transfer. The Lotus Connector for Notes and Lotus Connector for DB2 support array transfer for insert operations. The Lotus Connector for Oracle 7 supports array transfer for fetch and insert operations. The other Lotus Connectors do not support array transfer. • Creating Indexes for RDBMS A writeback index, also known as a database index, can accelerate LEI performance for RDBMS that support writeback. A writeback index enables writeback support for a given metadata object. Some Connectors require that a unique index exist, while others do not. The “Create Connection A Metadata” and “Index to create on Metadata Creation” options on the Replication Activity document pertain to index creation. The index created is used by LEI when performing data comparisons during replication. An index is crucial for good performance against some Connectors, such as relational databases. The Lotus Connectors that support a writeback index are Notes, DB2, ODBC, Oracle 7, Oracle 8, and Sybase. • Refresh Database Some Connectors and Activities support automatic or requested database refresh. Database refresh reflects the database changes in the database views at the end of the transfer or replication process when the destination database is being closed. Since views can be slow to re-compute following many changes, enabling this option is not advised. If enabled, the transaction must wait to complete until the database is completely regenerated. It also causes the refresh to occur as part of the transfer rather than when you next open the view. The next task can’t start until the transaction is complete and the database is regenerated. Enabling database refresh increases transaction times significantly, thus slowing down processing performance. 14 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Advanced RealTime Activity: Indexing Increases Performance High performance is crucial for the Advanced RealTime Activities, since they may be serving many clients. Performance can be increased by indexing fields on the external system. This is true for both Virtual Documents and Virtual Fields. The following two sections bullets discuss the benefits of indexing fields to increase performance. However, more detailed information can be found in the Virtual Documents activities chapter of this guide. • Performance in Virtual Documents Activities and Virtual Documents Using Virtual Attachments It is critical to the performance of the Virtual Documents Activities to build indexes on the external system server. See “Required External System Fields for Virtual Documents Activities” and “Usage Requirements for Virtual Documents and Virtual Attachments” in the “Virtual Documents Activities” chapter, for complete information on the issue of indexing when you use Virtual Documents activities. If you are using Virtual Attachments with Virtual Documents, certain fields in the Virtual Attachment Table should be indexed. For information concerning indexing Virtual Attachments fields when used with Virtual Documents, see “Virtual Attachments in Virtual Documents” in the “Virtual Documents Activities” chapter. • Performance in Virtual Fields To maximize performance in Virtual Fields Activities, when the external Connector is an SQL database, make sure an SQL index exists on the key fields. For more information, see “Usage Requirements and Performance in Virtual Fields Activities” in the “Virtual Fields Activities” chapter. Using Advanced RealTime Activities in a Clustered Domino Server Environment Advanced RealTime Activities can be used to great advantage in a clustered Domino server environment. See Chapter 1 of the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for complete information. Chapter 1: Introduction 15 Web-based Application Considerations If you are using a Web-based application and programmatically connecting to a database that supports synchronous or asynchronous data transfers (such as DB2 or Oracle), set the data transfer method to be asynchronous. LEI on the iSeries This section provides information about LEI specific to its use and implementation on the iSeries platform. LEI Server on iSeries On iSeries, the LEI server is a 64-bit application which will execute as an addin server task to a native 64-bit Domino server on iSeries. Access to DB2 databases, Notes databases and File System objects is allowed via the LEI server running on iSeries. Direct (native API) Connections to databases such as Oracle and Sybase are not supported on iSeries, nor is the ODBC Connection allowed. Access to non-DB2 data sources can be achieved by using the IBM DataJoiner product on an NT IPCS server, outboard NT, or AIX platform to reroute DRDA requests from iSeries to the target data source. Connections to Sybase, Oracle and other ODBC accessible databases can, however, be registered in an administration database for LEI servers configured to run on non-iSeries platforms in your LEI cluster. LEI Server Running Window on iSeries Unlike other LEI platforms, iSeries does not have an LEI server window or console. The running state of LEI can be viewed from the LEI administration database or from the Domino console on iSeries (through Show Tasks). The LEI server is an addin task to iSeries Domino server and so it is not launched in its own interactive process or window. Commands (such as Shutdown Server, Stop Server, Close Activity, and Kill Activity) that can be issued in the server window on other server platforms are available from the Action menu of the LEI Administrator or the Domino console. The Reconfigure option is automatic. When the LEI server configuration document is changed, the corresponding server will reconfigure during its next polling cycle or when the server is restarted, whichever happens first. 16 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Languages Support on iSeries LEI is considered an administrative tool and is therefore currently available in English only. The LEI user interface is available in English only. The English language (2924) does not have to be installed on your iSeries. Code Page Translation Support on iSeries Data translation is automatic (not optional) for LEI on iSeries because iSeries uses EBCDIC character sets. This is incompatible with Domino’s use of the LMBCS character set. LEI supports data translation in a variety of language code pages. LEI Activities launched under the LEI server will default to the job CCSID of the Domino server. See iSeries Domino server documentation for how to set the default job CCSID (the locale information specified in the QNOTES user profile) for Domino server addins. All LEI Activity functions are supported for iSeries. The following Lotus Connectors and Metaconnectors are supported for iSeries: • Connectors: DB2, Notes, File, Text • Metaconnectors: Collapse/Expand, Meter, Order, Trace Term Comparison for iSeries The following table compares key terms that you will find in this manual. Term iSeries Relational DB SQL Notes Database Any RDB registered via WRKRDBDIRE, including the local DB2/400 database Database Database (.nsf file) Metadata File Table Form Record Record Row Document Field Field Column Field Statement n/a Query Selection formula Chapter 1: Introduction 17 Chapter 2 Starting and Running the LEI Server This chapter provides information and instructions for starting and running the LEI server on the supported operating systems. Starting the LEI Server The LEI server must be running in order to execute LEI Activities. Start the LEI server by executing the LEI program at the LEI server machine. The LEI program name varies by platform. Platform Name Win32 nlei.exe AIX lei Solaris leis iSeries lei On Windows, you can also start the LEI server by selecting it from the correct program group folder using the Start menu. The LEI console appears, displaying the LEI server commands. When started, the LEI server connects to the LEI Administrator database and immediately runs any overdue Activities. It continues to poll the LEI Administrator at the interval defined in the server configuration document and to run scheduled Activities until shut down. 19 Executing LEI Server Commands From the Console The following LEI server commands can be executed from the server console window. The LEI server console commands are described below. Command Description Help Displays a brief description of the LEI server commands. List Displays a list of Activities that are currently being executed by this LEI server. The number associated with each Activity name must be used with the Close and Kill commands. Status Displays basic configuration information about this LEI server, as well as server time and status. Quit Shuts down the LEI server and all LEI Activities. Note When issuing commands in the LEI console, the command text is limited to 255 characters. Additional Activity and server control options are available on the Actions menu using the LEI Administrator under the LEI Server Administration and Activity Administration menus. See the “Executing LEI Server Commands from the Administrator” section in this chapter for more information. Executing LEI Server Commands from the Administrator The following LEI server commands can also be executed from within the LEI Administrator. • Close Activity • Kill Activity • Shutdown Server • Stop Server To close or kill an activity, choose Actions — Activity Administration, and choose either Kill Activity or Close Activity. To shutdown or stop the server, choose Actions — LEI Server Administration and choose either Shutdown Server or Stop Server. In either case, if an inappropriate document is selected when the action is attempted, an error message appears. If you receive an error message, see the Error Messages and Troubleshooting appendix. 20 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Running LEI as a Domino Addin Task You can run LEI as a Domino server addin task. This option is available when you install LEI on Win32 and UNIX. LEI on the iSeries is installed as a Domino addin task. This method automatically starts LEI when the Domino server starts and shuts down LEI as the Domino server is stopping. If you did not select this option during install, you can enable it manually by editing the Domino server’s notes.ini file and adding the task “lei” to the ServerTask list. LEI can also be manually controlled as an addin task. Note When used as a Domino addin, LEI converts all date/times to US format on the Domino console. Note If the Domino server starts as an NT service, to avoid sequencing problems LEI should be added as a Domino server addin task. This will ensure that Domino will start before there is an attempt to start LEI. To start LEI as an addin manually, enter the following command from the Domino server console: load lei addin To manually shut down LEI when running as an addin task, enter the following command from the Domino server console. tell lei quit When running as an addin task, the LEI console display is suppressed. To send LEI console commands, type the following command in the Domino server console: tell lei <command> For example, the following command will have the same effect as entering “l” at the standard LEI server console: tell lei l Considerations When a Notes/Domino server is started as an NT service, Notes/Domino users cannot access database/directory links that are at locations other than the local NT server. Applications installed as a service begin execution before the user is prompted to log in at the NT server. Therefore, when Notes/Domino starts up as an installed service, any mapped drives that the NT server may have set up are not yet reconnected at the point when the Notes/Domino server is started. The Notes/Domino server only recognizes drives that are available when it is first started. Drives that are mapped after the Notes/Domino server has started are not recognized. In this Chapter 2: Starting and Running the LEI Server 21 scenario, directory/database links that point to mapped drives on other computers will not function properly if the Notes/Domino server is started as a service. The solution for this set of circumstances is to not install Notes/Domino as a service. When Notes/Domino is not installed as a service, but rather is manually initiated from either the Start Menu or a desktop shortcut, you must be physically logged in at the NT server. Because you must log in at the server, all network drive mappings will have been established when the Notes/Domino server is launched. Using this method, directory/database links that are beyond the local server will function correctly. Understanding the leiclean Script for UNIX and iSeries The leiclean program is provided with LEI installations on UNIX and iSeries platforms to clean up LEI resources following an abnormal termination of LEI. • UNIX leiclean • iSeries call Qnotes/leiclean Use the leiclean program with caution. It kills all LEI processes owned by the user who executes it. It also frees all shared memory and semaphores currently in use by that user. The removal of shared memory and semaphores could affect other programs that are running under the same user ID, including the Domino server and any other Notes API programs. If you execute LEI and the Domino server under the same user ID (as you must to use Advanced RealTime), an abnormal termination by either process may halt the other. This is a Notes API limitation. For Unix users, in addition to leiclean, there is a shell script that can be used to clean up Notes/Domino processes after an abnormal termination. To use this, change your directory to the Notes data directory and then execute the following: nsd -kill 22 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Starting and Running the LEI Server on iSeries There are two ways of starting the LEI server after it is installed: automatically or manually. Starting the LEI Server Automatically When you install LEI on iSeries, the notes.ini file is automatically updated to configure LEI as a Domino addin task using the ServerTask environment variable. Configuring LEI as a standalone is not supported on iSeries. Starting the LEI Server Manually If you close the LEI server process by issuing the command “tell lei quit”, from the Domino console, you can restart the LEI server manually using the Work Domino Console (WRKDOMCSL) command. For example, to work with the console for a Domino server called LEISERVER1, you would enter the following at the Domono console command line: WRKDOMCSL LEISERVER1 From the console, you can also the following Notes command to start the LEI server: LOAD LEI Note With either method, if the LEI Administrator database (decsadm.nsf) resides on a different Domino server, that server must be active before LEI will start. Stopping the LEI Server on iSeries There are three ways to stop the LEI server. • From the Domino console • From the LEI Administrator • By stopping the Domino server From the Domino Console From the Domino console, you can end the LEI server by issuing the following command. tell lei quit If you have LEI Activities currently running, they will end as soon as they are notified of the quit request. The Activities will not run to completion. Chapter 2: Starting and Running the LEI Server 23 From the LEI Administrator You have two options from the LEI Administrator using the *RUNNING view: • Shutdown — Ends LEI in an orderly fashion; Activities run to completion • Stop — Ends LEI immediately; Activities do not run to completion By stopping the Domino Server By issuing an ENDDOMSVR server name command, the LEI server will also end in a controlled manner. If you have LEI Activities currently running, they will end as soon as they are notified of the server stopping. The Activities will not run to completion. If you stop the Domino server using the *IMMED option, the jobs end immediately and any LEI Activities currently active are terminated. This can cause unexpected results, such as incomplete transfers or lengthy rollbacks. You should only use the *IMMED option if all other methods, including *CNTRLD, fail. Determining if LEI Server is Running on iSeries There are three ways to determine if your iSeries LEI server is running. • Show Task from Domino Console • WRKACTJOB on iSeries • From LEI Administrator Database Show Task from Domino Console • WRKDOMCSL domino-server-name • show tasks • F5 — REFRESH The following tasks may display the activity name in the description. • LEI server — Administrator is running • LEI server — Server is running • LEI server — Activity Transfer Daily Issues is running 24 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide WRKACTJOB on iSeries When you enter the command WRKACTJOB, you will see the following LEI jobs under the subsystem for the Domino server. If Activities are running, there will be an LEIACT job running for each running Activity. DOMINOxx ... ... LEICSM LEI From LEI Administrator Database Select the *RUNNING view and look at the active server to see the list. Chapter 2: Starting and Running the LEI Server 25 Chapter 3 LEI Administrator This chapter introduces the LEI Administrator and describes its usage. LEI Administrator The LEI Administrator is a Notes application. The LEI Administrator database (decsadm.nsf) contains all of the configuration, activity, connection, and scheduling elements that LEI servers use to collect, transfer, and write data. Additional information about the LEI Administrator can be found in the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide (leiig.nsf and leiig.pdf supplied with LEI and also at http://www.notes.net). The LEI Administrator interface is shown in the figure below. The Navigator is shown on the left. The LEI Administrator enables you to view, create, and manage connections and activities. This chapter documents its features and behaviors. 27 LEI Administrator Database The LEI Administrator database (decsadm.nsf) is derived from the supplied template file leiadm.ntf. It is installed as part of the LEI installation procedure. At the time of installation, the design of an existing decsadm.nsf is upgraded to the LEI database, preserving existing connection and activity documents. This prevents the user from using both DECS and LEI concurrently on the same Domino Server. See the Lotus Enterprise Integrator for Domino (LEI) Installation Guide for more information on configuring LEI and about migrating documents from an existing Release 3.n LEI Administrator to this release of LEI. A single LEI Administrator can hold information used by several LEI servers, but each LEI server can be governed by only one LEI Administrator. A group of LEI servers governed by one LEI Administrator is called an LEI cluster. The LEI Administrator database interacts with a log database that documents the processing and outcome of activities and server operations. This is described later in this chapter. Note Do not use Notes replication on the LEI Administrator database. LEI server operations such as scheduling and status checking will be seriously disrupted. Configuring Administrator Security The LEI Administrator uses security features available with Lotus Notes, such as encryption and true Access Control List (ACL) author access. Since several types of users can use LEI, you should carefully plan your security implementation. Lotus recommends that one person, or group of people, be designated as the LEI Administrator manager. This person or group should be responsible for installing LEI and editing configuration documents. Using the security features of Lotus Notes (such as ACL author-level privileges), these documents can be made inaccessible to other users. Note The default access on the supplied LEI Administrator is Manager. Each LEI customer’s database manager or system administrator is urged to restrict this Manager access to the appropriate (typically small) group of users during installation or immediately after installing LEI to prevent unauthorized user access to the database. Additional managers can be added at or after LEI installation by adding entries to the access control list. See the Lotus Enterprise Integrator for Domino (LEI) Installation Guide for related information. 28 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Encryption of database passwords is an important part of LEI’s security features. All password fields are encryption enabled. Encryption keys for each form must be made and added to each LEI server’s ID as well as to anyone’s ID who may need to see or make changes to the encrypted fields or to the document itself. Those users who do not have the encryption key may still view the document but will not see any data in the encrypted field. The encrypted field remains blank. Additionally, users without the encryption key will not be able to edit the document. Configuring LEI password encryption is described in the Lotus Connectors and Connectivity Guide (lccon6.nsf and lccon6.pdf supplied with Domino and also available at http//www.notes.net). Also, encryption in Advanced RealTime Activities, such as Virtual Fields, Virtual Documents, and Virtual Agents, are documented in this manual in its respective activity chapter. The LEI Administrator provides author privileges at the document level. Each LEI document has an action button in the action toolbar that accesses the author privileges list. Another part of security is reader access. When the encryption button is used in a document, the first part of the dialog displays the reader access list. Disabling the option “Who can read this document: All readers and above” and picking names from the list allows only those names to see the document. No one else will be able to view the document, including the document’s author, if their name has not been chosen as part of the list. Please see the Administrator’s Guide for Lotus Notes and the Database Manager’s Guide for more detailed information on Lotus Notes security features. LEI Administrator Action Bar The LEI Administrator contains the buttons you need to create connections and activities. It also contains buttons you can use to start and stop activities and display the current activity log. Note See the section entitled, “Obtaining Help and Opening User Documentation” in Chapter 1 of this guide to learn how to obtain user assistance as you are creating, editing, and deleting activity and connection documents. You can select the Create button in the top bar as an alternative to the Add Connection and Add Activity action buttons. You can also select Chapter 3: LEI Administrator 29 the Actions button in the top bar as an alternative to the Stop Activity action button. The LEI Administrator action bar buttons are described below. • 1. Add Connection — Creates a new Connection Document for an external data source. When you click the button you are presented with a list of possible connection types. Make your selection and a new Connection Document appears. You can optionally enable the User Assistant to open the Connection Document with explanatory text at the top of the document. Creating connections is described in the companion document Lotus Connectors and Connectivity Guide. Popup help is also available on the document. The available connection types are DB2, File, Notes, ODBC, OLE DB, Oracle, Sybase, Test, Other (Collapse/Expand Metaconnector, Meter Metaconnector, Order Metaconnector, and Trace Metaconnector). The User Assistant toggle is located in the Help area of the Navigator panel. • 2. Add Activity — Creates a new Activity Document. When you click the button you are presented with a list of possible activity types. Make your selection and a new Activity Document appears. If you wish to create one of the three Advanced RealTime activities (Virtual Agents, Virtual Documents, and Virtual Fields) you can optionally enable the User Assistant to prompt you through the creation process. Creating activities is described in this guide. Popup help is also available. The available activity types are Archive, Command, Direct Transfer, Java, Polling, Replication, Scripted, Virtual Agents, Virtual Documents, Virtual Fields, Admin Backup, and Admin Purge Log. • Start Activity — Starts the Activity that is currently selected in the Activities view below the action bar. This has no effect if the selected activity is already running. • Stop Activity — Ends execution of the currently selected Activity. This has no effect if the current selection is not running. • Current Activity Execution Log — Displays the status of the currently selected Activity. If the selected Activity is running, the start time and current status are displayed. If the selected Activity is not running, the results of the most recent execution are displayed. 30 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide You can optionally view all Activity logs by clicking the Log option in the Log History area of the Navigator panel. LEI Administrator Navigator and View Options The Navigator panel on the left side of the LEI Administrator enables you to view activities, connections, and configuration documents in the viewing area of the LEI Administrator. It also allows you to group activities and/or connections using user-defined folder names. The Help section of the Navigator enables you to toggle the User Assistant on and off and quickly open the online documentation. The Navigator also allows you to quickly display log history by server and type. Chapter 3: LEI Administrator 31 The Navigator panel options are described below. • Activities — Using this option, you can view activities in one of the following four possible ways. The Activity Documents can be opened and edited from this view. An activity view also displays activity name and status, such as “Completed successfully” or “Has not run”. • By Type (default) — Select this option to display all activities, grouped by activity type, in the activities view. • By Category — Select this option to display all activities and connections that contain a shared Category name, grouped by that Category name, in the activities view. Connection Documents, Configuration Documents, and Activity Documents contain an optional Category field that enables you to enter a name that can later be used to group documents by that Category name. 32 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • By Advanced RealTime — Select this option to display the following grouped activity types in the activities view: Virtual Agents, Virtual Documents, and Virtual Fields. The information in the Advanced RealTime view displays the Domino Application (and the form used), the events that the Activity performs (such as, open, update, and so on), Activity type, the external source, the status of the Activity, and the name of the Activity. • By Data Management — Select this option to display the following grouped activity types in the activities view: Archive, Command, Direct Transfer, Java, Polling, Replication, Scripted, Admin Backup, and Admin Purge Log. Chapter 3: LEI Administrator 33 • Connections — Select this option to display all connections, grouped by connection type. The Connection and Metaconnection Documents can be opened and edited from this view. Both the Name and External Source columns can be sorted by the user in order to easily locate documents. • Configuration — Select this option to display the configuration documents for the LEI server and the LEI Administrator. You can open and edit these documents as required. There is one server document and one administrator document. For more information about server and administrator configuration documents, see “LEI Configuration Documents” later in this chapter. 34 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • Project Organizer — Select this option to see the Project Organizer view. Using this view, you can optionally create folders, and place particular connections and activities in those folder, for the purposes of grouping by project or some other user intent. For example, you may wish to group connections and activities together that pertain to a specific task. You can create folders here by clicking Create — Folder in the Domino top bar menu. Use a name that reflects the kind of information you are monitoring. Once you have created a folder, drag and drop the desired connections and/or activities into that folder and then display the folder contents in the view. • Help — Using this option, you can toggle the User Assistant on and off, open the online documentation, and learn more about the LEI Administrator database itself. • Turn User Assistant On/Off — Click this option to enable or disable the User Assistant. When the User Assistant is on, Connection Documents open with explanatory text at the top. If you are creating a Virtual Fields, Virtual Documents, or a Virtual Agents Activity document, the User Assistant guides you through the process of creating that Activity Document. Chapter 3: LEI Administrator 35 • LEI Documentation — Select one of the LEI online manuals from the list to open the associated .nsf file. Navigate to the needed topic using the panel of topic contents in each guide. IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide (leidoc.nsf) Lotus Connectors and Connectivity Guide (lccon6.nsf) Lotus Connector LotusScript Extensions Guide (lsxlc6.nsf) IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide (leiig.nsf) These online manuals are also supplied in PDF format and can be found at http://www.notes.net. • About this Database — Select this option to open a document that describes how to use the Administrator. • Log History — Select the Log option to view all of the log file documents by server or type. You can optionally open and view these documents. Active View Option Similar to the Log History option in the Navigator, the Active View option, which appears at the bottom of the LEI Administrator underneath the Activities or Connections view, enables you to display anything that is currently running. The window can be expanded or collapsed as needed by clicking the heading to display the active servers and the currently running activities. You can double-click the Activity or Active Servers headings to expand or collapse the list. You can also double-click an entry to open the associated document or log history. The window allows you to stop a currently executing activity or view the activity execution log while the activity is in progress. 36 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Activity Status Indicators The following table describes the status indicators shown in an Activities view. These icons may appear in different locations depending on the Activity. To refresh an Activities view, press the F9 key. Column Description and Icon Status Shows the status of the Activity, as indicated by the icons below. Refresh the view periodically to update the status. Active Activity Starting / Stopping an Activity Activity stopped with an error Activity disabled with an error Activity scheduled to AutoStart with server Activity not scheduled Activity scheduled to start and stop at specified times Events Shows the type of event being monitored by this Activity. Used in the Advanced RealTime activities view. Indicates the Activity will process document creations Indicates the Activity will process document opens Indicates the Activity will process document updates (edit/save) Indicates the Activity will process document deletions Indicates Virtual Attachments are enabled in this Activity Activity Icon is displayed indicating whether the Activity is a Virtual Field or a Virtual Document Activity: Indicates the Activity is a Virtual Agents Activity Indicates the Activity is a Virtual Fields Activity Indicates the Activity is a Virtual Documents Activity Chapter 3: LEI Administrator 37 LEI Administrator Menu Commands In addition to the standard Notes commands found in the Domino top bar menu, the LEI Administrator provides additional commands. Create When you select Create from the Domino top bar menu, the following LEI-specific options appear. • Other — This option enables you to choose a Metaconnection type to create. You can also create a Metaconnection using the Add Connections button in the LEI Administrator. • Agent — This option enables you to create an LC LSX agent. • Folder — Used in conjunction with the Project Organizer option on the Navigator, this option enables you to create a custom-named folder for the purpose of grouping specific connections and/or activities. • View — This option enables you to define an activity or connection view. Actions When you select Actions from the Domino top bar menu, the following LEI-specific options appear. Activity Administration The Activity Administration menu provides commands related to activities. Command Description Clear Lock The ClearLock agent clears an Activity document’s Lock field so that the Activity can be run again. The purpose of the agent is to provide a manual way to clear the lock in case an LEI server goes down while running the Activity and you want to run the Activity before the automated server cleanup occurs. Select the documents from a view before running this agent. The agent ignores non-activity documents. Kill Activity This command immediately terminates an Activity process. Note The preferred method of stopping an Activity is the Stop (Activity) command, which provides an opportunity for the Activity to clean up its database connections and log the termination. Using the Kill command may leave the system in an unstable state due to improper termination of the Activity, so only use it when absolutely necessary. LEI considers killing an Activity an error. continued 38 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Command Description Set Designated Server Use this agent when a server configuration document has been unintentionally destroyed. Since LEI identifies servers through an ID rather than by name, all activities using the server field will be synchronized with the new server ID. The agent acts on selected activities. Stop Activity This command stops a selected Activity that is currently running. Stop (Activity) is the preferred method (rather than the Kill command) of prematurely terminating an Activity. When an Activity is closed, LEI does not consider it an error. LEI will not mark the Activity as failed, and does not send mail notification on error. The Activity is not completed and the Activity and dependent activities are not executed. Note To immediately close an Activity process, use the Kill command. LEI Server Administration The LEI Server Administration menu provides commands related to the operation of the LEI server. These commands are described below. Command Description Shutdown Server This command terminates the LEI server. Shutdown Server is the preferred method (over the Stop Server command) of prematurely terminating a server. It prevents the LEI server from starting any new activities and waits for all currently running activities to shut down. When all activities have shut down, it stops the LEI server program. To immediately terminate the LEI server, use the Stop Server command. Stop Server This command immediately terminates the LEI server. It prevents the LEI server from starting any new activities and sends a Stop (Activity) command to all currently running activities. After the time period specified in the server configuration document’s Activity Request Shutdown Timeout field (in the Administrator database), any activities that are still running are Killed (see the Kill command) and the LEI server program stops. Note The preferred method of terminating the LEI server is the Shutdown Server command. Chapter 3: LEI Administrator 39 Reset Activity This command restarts the currently selected Activity. Set Autostart When this option is turned on all the selected activities start when Domino starts. Clicking “Off” turns off this option for the selected activities. Remove from Folder Remove the selected Activity and Connection Documents from the currently opened folder. This option is only available when you are in a Project Organizer folder. Creating, Editing, and Deleting Documents You can create Connection Documents, Activity Documents, and Metaconnection Documents using the LEI Administrator. Note Administrator and server configuration documents are created when LEI servers are installed. See “LEI Configuration Documents” in this chapter for more information about administrator, server, and client configuration documents. Note See the section entitled, “Obtaining Help and Opening User Documentation” in Chapter 1 of this guide to learn how to obtain user assistance as you are creating, editing, and deleting activity and connection documents. Creating New LEI Documents This section provides a very brief overview of the process for creating Connection and Activity Documents in LEI. This process is documented in great detail, per each activity type and connection type, in the companion document Lotus Connectors and Connectivity Guide and this guide. To create an LEI Connection Document or Activity Document, do the following: 1. Start Lotus Notes. 2. Open the LEI Administrator (decsadm.nsf). 40 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 3. Click the Add Connection or Add Activity icon from the action bar. Alternatively, you can select the Create button from the Domino top bar. Note You cannot create an activity until the needed connection documents have been created. 4. Choose the kind of document that you want to create (for example, DB2 Connection or Direct Transfer Activity. 5. Enter information into the document as needed. Note You can select from a list where multiple choices are possible. Click the down arrow icon to display the fields available in the available external data. This is referred to as “browsing”. The external metadata is queried each time that you click the down arrow icon, therefore no refreshing of the cache is required as in previous releases of LEI. In most cases, you may also type entries in manually, however, using both manually typed entries and browsed entries may result in unpredictable results or error messages and is not currently supported. If you choose to create the activity using the “browsing” feature, use this feature throughout the document. If manual entry is preferred, manually type throughout the document. See the section entitled, “Understanding Basic Browsing” later in this chapter for proper configuration of connectivity to external data sources. When completing the fields, follow these guidelines: • Some entries are made by selecting from a list. The labels of these lists are followed by the standard Notes entry helper button. Click the arrow to the right of the field to see the key list and click the selection that you want to make. • Small blank buttons on the form provide access to additional options when clicked. • Red field brackets indicate encryption-enabled fields. • All documents contain an optional Category and Comments fields that can help you manage your documents. Category — Enter a category name. This option enables you to group a set of documents by like category name. If the category name you enter doesn’t exist, it will be created when you save the document. You can enter multiple category names, separated by commas. Comments — Enter text to describe the document. This is a rich text field, you can place anything here, including links to other documents. Chapter 3: LEI Administrator 41 6. You can use Notes Reader and Author list privileges to add additional security to Activity and Connection Documents. To add or update authors of the document, click the Author Privileges action button for a list of three options for managing Author access. Select an option and click OK to use it. Document authors are people who will have authorization to edit the document once it has been created. Privileges and ACL information for LEI usage is thoroughly documented in the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide. • Add name[s] to document editor list. Select this option to add a name(s) from the Domino Directory (formerly referred to as the Public Name and Address Book) to the list of editors for this document. • Remove a name from document editor list. Choose this option to delete a name from the list of document editors. • View document editor list. Choose this option to view the list of current editors for the document. 7. Save the completed form by clicking the Save & Exit button at the top of the form. Alternatively, you can choose File - Save from the top bar menu. Editing Existing Documents To edit an existing Activity Document or Connection Document, do the following: 1. Open the LEI Administrator (decsadm.nsf). 2. Choose a view from the LEI Navigator. 3. When the document appears, double-click the name of the document in the view. 4. Enter the Edit mode using one of the following methods: • Press the Ctrl + E keys. • Choose Edit Document from the Action menu. • Choose the Edit Document button at the top of the form. • Double-click anywhere in the document. 5. Make the desired changes to the document. 6. Save the document by clicking the Save & Exit button at the top of the form. Alternatively, you can choose File - Save from the top bar menu. 42 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Deleting Documents When you no longer need a document, you can delete it, provided that you have appropriate access rights. Note When deleting Connection Documents, you must know whether or not they are used by other documents. You can sort Activity view by Connection by clicking the Source or Target Connection column heading in the Activity view. Deleting a document whose name remains in another document will cause an error when that Activity is run or when you try to open a dependent document. Note Do not delete configuration documents; they should be deleted only by the LEI install or uninstall utility. To delete an Activity Document or Connection Document, do the following: 1. Open the LEI Administrator database (decsadm.nsf). 2. Choose a view from the LEI Navigator, for example Connections. 3. Select the document you want to delete and press the Delete key. The document will be deleted when you close or refresh the view. Alternatively, on a Win32 platform you can select the document and press the Ctrl + X keys to immediately delete the document and its display from the view. Activity Document Action Bar LEI Activity, Connection, Metaconnection, and Configuration Documents may include the following action buttons when they are opened for viewing or editing. Other buttons, specific to a given activity or connection type, may also be present. Such buttons are documented with the applicable activity or connection type. • Author Privileges • Edit Document • Start! • Save & Exit Chapter 3: LEI Administrator 43 Author Privileges The Author Privileges option allows you to change author privileges for the document currently being edited. You can add additional security to Activity, Metaconnection, Connection, and Configuration Documents. Document editors are people who will have authorization to edit the document once it has been created. To change Author Privileges for the Activity, perform the following tasks: 1. Click the Author Privileges action button. The Author Privileges dialog box appears. 2. Choose the action you want from the options available in this dialog box. These actions include: a. Add name[s] to document editor list. Choose this option to add a name(s) from the Notes Public Name and Address Book (Domino Directory) to the list of editors for this document. b. Remove a name from document editor list. Choose this option to delete a name from the list of document editors. c. View document editor list. Choose this option to view the list of current editors for the document. Edit Document Selecting this action button opens the current document for edit. It also toggles the Save and Exit action on. Start! Selecting this action button causes the Activity to be run the next time the LEI server polls the LEI Administrator database. In previous releases, this button was named Run ASAP. Click the Start! button when you want to run an Activity as soon as possible. LEI runs the Activity the next time the LEI server polls the LEI Administrator. The Start! button is available only from an Activity Document after the document has been saved. 44 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Save and Exit Selecting this action button saves the changes that you have made to a document. Understanding Basic Browsing Earlier releases of LEI used client-side browsing from the LEI Administrator, which required that LEI users install their external system connectivity software on both the Domino server (that hosted the LEI Administrator and the LEI server) and the Notes client. This is no longer true. LEI now uses server-side browsing. Browsing databases on the external system uses client connectivity from the Domino server that hosts the LEI Administrator. External system connectivity software should be installed on the Domino server that hosts the LEI Administrator and that also hosts the LEI server. If your Domino server and LEI server are on different machines, you must install your external system connectivity software on both machines. LEI users no longer need to install external system connectivity software on the Notes client. Browsing and the Domino Directory Certain security settings are required in the Domino Directory where the LEI Administrator resides. In the Security tab of the Domono server’s Server Configuration Document, the “Only allow server access to users listed in the directory” option is set to “Yes” by default. However, to browse the server, set the “Only allow server access to users listed in the directory” option to “No.” Ensure that all users who will be creating LEI Activities have entries in the Domino Directory and that the server is listed as part of a group. For more information, see the pop-up help in the Server Name and Address document. Other Domino Directory setting requirements are described in the Lotus Enterprise Integrator for Domino (LEI) Installation Guide in the introductory chapter and in the “Preparing to Install LEI on Windows” and “Preparing to Install LEI on UNIX” sections of its Windows and UNIX chapters, respectively. Chapter 3: LEI Administrator 45 Buffering Lists When the list of tables is lengthy, you can scroll down the list to find the table you want to select. However, if the table list is too long to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>). A box also appears under the list, which you can use to enter the exact name of the table you want to use. See the illustration below. Note The number of tables that the scrolling window can accommodate is dependent on the number of characters in the table names. If table names are short, more tables will be viewable by scrolling than if table names are long. 46 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Configuring Browsing To enable browsing of Notes databases on a remote Domino server, you must edit the names.nsf file (Domino Directory — formerly referred to as the Name and Address Book or NAB) that resides on the remote Domino server. This procedure is thoroughly documented in the Lotus Enterprise Integrator for Domino (LEI) Installation Guide in Chapter 2, “Installing and Configuring LEI on Windows” and Chapter 3, “Installing and Configuring LEI on UNIX” in the following section titles: • Setting Access Controls in the Domino Directory • Establishing Access Rights in Your Notes ID File LEI Configuration Documents There are two LEI configuration documents available; Administrator and Server. These documents define parameters for the LEI Administrator databases, LEI client databases, and LEI servers, respectively. These documents are accessible using the Overview option in the LEI Navigator or using the View menu under System Overview. Administrator Configuration Each LEI Administrator database contains a single Administrator configuration document. The Administrator configuration document contains configuration information about the Administrator and the LEI servers. Chapter 3: LEI Administrator 47 The Administration Configuration document identifies the following information. Field Name Description Log Database Specifies the file path and name of the LEI log database on the Domino server, relative to the standard data directory. Help Database Specifies the file path and name of the LEI help databases on the Domino server, relative to the standard data directory. Scripted Agent Database Specifies the file path and name of the optional repository for Scripted Agents, relative to the standard data directory. Status Broadcast Interval Specifies the interval, in minutes, at which each running LEI server must notify the Administrator database of its status. Any LEI server that fails to update its status in three of these time periods is assumed to have terminated abnormally. The interval value must be entered in minutes and defaults to a value of 60. The status of the server is reported in the server configuration document along with the date and time of the last broadcast received. Server Configuration The LEI Administrator contains a server configuration document for each LEI server that it controls. The document contains configuration information relevant to the LEI server. An example is shown below. The server configuration document identifies the following information. Note Changes made to the LEI server’s configuration document (such as Poll Interval, Maximun Duration of Activities, and Activity Shutdown Request Timeout) do not take effect until you restart the LEI server. 48 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field or Button Description Current Status Specifies the status of the server. This information cannot be modified directly. The status is Active or Inactive. Last Broadcast Specifies the date and time that the Administrator last received a status update from the server. Like the Current Status, this information cannot be modified. The frequency of broadcasts is the one entered in the Administrator configuration document. View log (button) Click the view log icon (if available) to see the latest log document for this server. Server Name Specifies the name of the LEI server given at install time. This information cannot be modified. See the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for more information. Poll Interval Specifies the interval at which the LEI server polls the Administrator database to see if there are any Activities that it needs to execute. Maximum Number of Activities Specifies the greatest number of concurrent Activities that the server will run. Once the server is running the maximum number of Activities, it postpones additional Activities until existing Activities are concluded. Zero stands for no maximum. Maximum Duration of Activities Specifies the longest duration, in minutes, that the server will run any single Activity. The server automatically closes any Activity that exceeds this duration. Zero stands for no timeout. Maximum Consecutive Failures Specifies the greatest number of consecutive failures that a single Activity can have before the server no longer schedules it for execution. Zero stands for no disabling. Activity Shutdown Request Timeout Specifies the amount of time in seconds that an Activity has to respond to a Close command before it is terminated. Zero stands for no timeout. continued Chapter 3: LEI Administrator 49 Field or Button Description Domino Server Name of server that hosts Domino. It may be the same as or different from the server that LEI has been installed on. Advanced RealTime Enabled: Mark this box to notify LEI that the specified Domino server can run Advanced RealTime Activities. Note See the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for more information about how to setup a Domino server so that it is enabled for Advanced RealTime Activities. LEI Log LEI maintains a log for all LEI servers and activities. The LEI log is useful for tracking performance and for troubleshooting. To view your LEI log, select Log from the Navigator, then double-click the specific entry you want to look at. Server Log The following information appears in the server log document: Field Description Server Specifies the name of the LEI server. (Date/time interval) Specifies the date and time that the LEI server was started and terminated, marking the duration of the server’s Activity during this log entry. The date/time format depends on the current Notes client operating system. Individual line items Includes the history of processing undertaken by the LEI server, including specific activities with their start and stop times, server status, and any error messages that may have occurred. 50 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Activity Log Shown below is an example of an LEI Activity log. Activity logs provide information about LEI activities that enable you to track system performance and results. The information in an Activity log includes the name of the Activity, the server that processed it, the start and finish date and time, and a summary of statistics. The following information appears in the Activity log document: Field Description Activity Specifies the name of the Activity. Processed by Server Specifies the name of the LEI that ran the Activity. (Date/time interval) Specifies the Activity’s start and end date and time. The date/time format depends on the current Notes client operating system. Individual line items Includes the history of the Activity’s processing, including event and error messages. Elapsed Time Specifies the duration of the Activity in the format hh:mm:ss. Records Fetched Specifies the number of records that were fetched from the external system. Records Inserted Specifies the number of records that were inserted into the external system. Records Updated Specifies the number of records that were updated into the external system. Records Removed Specifies the number of records that were deleted from the external system. Chapter 3: LEI Administrator 51 Operation Log The operation log holds any LEI system reports. LEI will only create one of these reports when an error has been detected within the LEI Administrator. As an example, if the scheduling information for an Activity were corrupted, then an operation log would be created to point out the Activity and the affected data. An example of an operation log is shown below. The following information appears in the operation log document: Field Description Generated for Server Specifies the name of the server for which the log was generated. (Date/time interval) Specifies the date and time that the events logged took place. (Individual line items) Lists any operational problems. 52 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LEI Script Vault The LEI Script Vault (leivlt6.nsf) is supplied with LEI. You can choose to create it when you install LEI. It can be used to store the LC LSX scripts (agents) that you create for use with LEI. The option “Catalog All Agents”, available at the View level, inventories the agents and creates a tracking document for each. This tracking document contains information about each agent, such as its name and creator, and allows you to add additional comments. When you open the LEI Script Vault, select “Catalog All Agents” to update the agent tracking document so that agents appear in the view just like documents would. Each time you select the “Catalog All Agents” option, the agent tracking document is updated, new agents are cataloged, and tracking documents that no longer have associated agents are deleted. The LEI Script Vault is supplied with two agents for use with the the supplied sample database leipackagetrack.nsf. The Tutorials appendix of this guide describes how to create a Scripted Activity using these supplied agents with the sample database leipackagetrack.nsf. The LEI Script Vault should be included in regular backup procedures. Note Only the agents in the LEI Script Vault (leivlt6.nsf) that have been created as shared agents are copied during the Admin Backup Activity. The Admin Backup Activity does not copy private agents from leivlt6.nsf to the target database. To ensure that your agents are copied to the target backup database during the Admin Backup Activity, enable the Shared Agent option in Domino Designer when you create your agents. Chapter 3: LEI Administrator 53 Chapter 4 Introduction to Connectors Several Lotus Connectors are supplied with LEI. Lotus Connectors Supplied with LEI The following Lotus Connectors are supplied with LEI: • Notes • DB2 • ODBC • Text • File • Sybase • Oracle 7 • Oracle 8 These Connectors are described in the companion document Lotus Connectors and Connectivity Guide. The documentation file names are lccon6.nsf and lccon6.pdf. The lccon6.nsf file is supplied with Domino, not LEI, but is reachable using the Help button on the LEI Administrator. The lccon6.pdf file is located at http://www.notes.net. Related information can be found in the Lotus Connectors LotusScript Extensions Guide. The documentation file names are lsxlc6.nsf and lsxlc6.pdf. The lsxlc6.nsf file is supplied with Domino, not LEI, but is reachable using the Help button on the LEI Administrator. The lsxlc6.pdf file is located at http://www.notes.net. For iSeries users, contact your system administrator to determine which Lotus Connectors are available. For more information about these and additional premium Lotus Connectors, visit the following Web site: http://www.lotus.com/ei 55 Chapter 5 Introduction to LEI Activities This chapter provides an introduction to LEI Activity Documents. It includes an introduction to Activity Documents, descriptions of fields common to all Activities, Activity scheduling options, definition of Advanced RealTime Activities, and information about command line execution of Activities. Introduction to Activity Documents Activity Documents contain the information that instructs the LEI server to execute an event, such as a data transfer. Each Activity type has a specific Notes form that can be used to create the Activity. Each LEI Activity is described briefly below. For more information, refer to the specific chapter for each Activity. Note Advanced RealTime consists of three Activities - Virtual Fields, Virtual Documents, and Virtual Agents. Activity Description Archive This activity archives a database. Command This activity executes an action against a database. Direct Transfer This activity transfers data from one database to another. Java This activity executes a user-specified Java application. Polling This activity polls a database to see if a specified condition exists, and, if so, executes an activity. Virtual Fields Advanced RealTime This activity provides real-time access from a Domino application to a supported external data source, such as DB2. Virtual Documents Advanced RealTime This activity provides functionality that is similar to Virtual Fields, however, it does not require key documents and make it possible to have a Notes database with a very small footprint access a very large amount of data from an external source. Virtual Agents Advanced RealTime This activity creates Domino agents that run stored procedures on the external system database. continued 57 Activity Description Replication This activity synchronizes data in different databases. Scripted This activity executes an activity whose data sources and function are defined by a LotusScript script. Admin Backup This activity backs up the LEI Administrator database data. Admin Purge Log This activity purges the LEI logs. Creating Activities The process for creating LEI Activities is described in the LEI Administrator chapter in the section entitled “Creating, Editing, and Deleting Documents”. It is also described at the beginning of each Activity chapter in this guide. While building an Activity, you can choose connection information from existing Connection Documents. You should create any Connection Document that you intend to use before you create the activity. See the companion document Lotus Connectors and Connectivity Guide for information about how to create or edit a Connection Document. This document is supplied with Domino, not LEI, but is reachable using the Help button on the LEI Administrator. The PDF equivalent is located at http://www.notes.net. All connections referenced in the Activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. Activity Documents use Notes collapsible sections to contain activity option sets. Most Activity Document fields are browseable, enabling easy point and click access. Browseable fields are not editable. Browsing Connections from within an Activity LEI supports server-side browsing. Browsing connections on an LEI server that resides on a Notes client, and not on the Domino server hosting the LEI Administrator, is not supported. If your LEI server is on a local client, and you want to connect to a local LEI database, you must type in the metadata and field names on the Activity Document. When browsing, the Domino server hosting the LEI Administrator searches for the database named in the Connection Document. If that database resides on a Notes client (not the Domino server hosting the LEI Administrator), LEI browsing cannot connect to the server to perform cataloging against it. 58 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide When using a Notes connection to a local database, browsing metadata and fields from the LEI activity yields the following error message: Cannot use a Notes connection to a local database - error: No such database exists Common Activity Fields and Buttons The fields and buttons described below are common to most Activity documents. Author Privileges Action Button This button appears in the Action Button Toolbar for all Activities. It allows the author of the document to view and change author privileges. Activity Name and Status The name and status fields that are common to all Activity Documents are described below. Field Description Author Displays the user name of the person who created the Activity. Name Specifies a user-defined name and serves as a unique identifier for the Activity. This uniqueness is enforced through a validation process when the Activity is saved. The character maximum is 127. Current Status Indicates the status of the document as reported by the server. The status can be any of the following: New — The Activity has not yet been saved. This status appears while the Activity is being built. Not Yet Scheduled — The Activity’s schedule has not been enabled since its creation. Scheduled for… — The Activity is enabled and scheduled to be executed at the time indicated. Active on… — The Activity is currently running on the listed server. Scheduling Disabled but Running — The Activity has been queued to run but has not started running yet. Disabled but Running — The Activity is disabled but is currently running as a result of being executed. Disabled — The Activity’s schedule is currently disabled. To enable the schedule, open the Activity document in Edit mode and change the Enabled/Disabled field to Schedule Enabled. continued Chapter 5: Introduction to LEI Activities 59 Field Description Last Computed Shows the time and result of the last run. Run Time Shows the date and time that the Activity last ran (successfully or unsuccessfully). A zero means that the Activity has never been executed. Result Indicates what happened in the last run. If the Activity has never run, you’ll see the caption “Activity has not yet run to completion.” Log Link Displays the log document. Click the Log document icon (if available) to see the log document for the last execution of this Activity. General Options The General Options fields let you define general Activity information. These options are described below. Field Description Designated Server Optionally specifies the LEI server. If no LEI server is specified here, the first server to poll the control store, when the Activity is scheduled to run, will execute the activity. If a server is specified here, only that server can run the Activity. Dependent Activity(s) Lists another Activity or Activities that will run when the current Activity executes successfully. With dependent Activity(s) you can connect the execution of two or more Activities. Activities must be defined (in their own Activity Documents) before they can be added as a dependent activity. continued 60 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description OS Priority Assigns a low, medium, or high priority to the Activity. In a multitasking environment, the LEI server allocates resources based on priorities. A high priority gains more LEI resources than a medium or low priority. Sets a time-limit in minutes for how long an Activity can run Maximum Duration of This before being terminated. Activity A setting defined here overrides the maximum duration time set in the server configuration for the server that executes the Activity. Zero stands for no timeout. Logging Enable Logging — If Enable logging is selected, an Activity log document is created for each execution of this Activity. When you select this option, the other logging options become available in the user interface. Buffer Logging — If Buffer Logging is selected, log entries are stored in memory until the conclusion of the Activity. When the Activity is complete, log entries are written to the log database. If Buffer Logging is not selected, log entries are written to the log database as they are generated. Send Log and Report Status Specifies whether mail should be sent Never, Always, or On Error. With the On Error option, mail is sent only when the Activity does not complete successfully. Send Log as Plain Text — By default, the log report and status are sent to the recipient in Notes rich text format with a doclink. Enabling this option sends the mail in plain text format instead. This option is provided for use when sending Email to a pager or PDA device. Email Recipients Enter the Notes user names of the people who should receive the Email notification. Use a comma to separate the names. Scheduling See the “Activity Scheduling Options” section later in this chapter. Chapter 5: Introduction to LEI Activities 61 Category and Comments The Category and Comments fields enable you to optionally categorize documents and add comments. Field Description Category Enables you to specify a category name. Entering a category name allows you to group a set of documents by category in the Activities by Category view. If the category you enter doesn’t exist, it will automatically be created. You can only enter a single category per document. Comments Enables you to add descriptive content. Use this field to describe the document. This is a rich text field, so that you can place anything here, including document links. Activity Scheduling Options The Scheduling section defines the schedule according to which the Activity is to be executed. Overview of Scheduling The scheduling section of Activity documents is common for all Activities. The parameters affect the scheduling in a specific order. Additionally, the Start Activity icon in the Navigator or the Run ASAP action button each mark the Activity to run as soon as possible regardless of the schedule. Neither of these affects the scheduled run time. It is important to note that the scheduling parameters affect when an Activity is scheduled to start. Additionally, for the Polling and Advanced RealTime Activities, the scheduling information is used to determine when it should stop. More information about how the scheduling settings affect the stop time may be found at the end of this section. 62 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide The first parameter of scheduling is the DISABLED/ENABLED/RESTRICT setting. When an Activity is disabled, all of the other scheduling settings are ignored. When enabled, the Activity will be scheduled to run according to the setting of the remaining parameters. The Activity may actually run some time after its schedule time if the LEI server is not available either because it is shut down or it is busy. In this case, the Activity will run as soon as possible after the server becomes available. If the Activity should not run except at the schedule times, the RESTRICT TO SCHEDULE should be used. The next parameters to affect the scheduled run time are the Start Date and Time and Stop Date and Time. These create boundaries for the run time. Scheduling will not begin before the start date and will not continue after the stop date. As an example, a start date of June 6, 1999 and a stop date of December 25, 1999 would prevent the Activity from being scheduled any time before 00:00:00 AM on the 6th of June and would prevent it from being scheduled any time after 12:59:59 PM on the 25th of December, 1999. If only a time value is indicated for these parameters, then the current date is assumed. Once the boundaries of scheduling are established, the Days of the Week, Weeks of the Month, and Days of the Month settings are used together to determine which dates in the month are acceptable. First, any dates which do not fall on the selected Days of the Week, Days of the Month, or Weeks of the Month are ignored. For the purposes of scheduling, a week is a seven day period starting with the first of the month. As an example, if the first of the month is a Thursday, then each week is Thursday through Wednesday. Both the Weeks of the Month and Days of the Month choices will accept negative numbers to indicate counting from the end of the month. This feature is helpful for scheduling Activities to run on the last day of the month or even the last Friday of the month. For example, to schedule on the last day of every month, set Days of the Month to -1; to schedule on the last Friday of the month, set Weeks of the Month to -1, Days of the Week to Friday. The final parameters for scheduling are Repeat Interval and Run at Times. When specified, any times which fall outside the Run at Times settings are ignored. The values may be any combination of distinct times such as 8:00 AM, 11:30 PM and time ranges such as 7:00 AM - 3:00 PM. Once the initial run time has been computed, subsequent run times are scheduled at the repeat interval. To maintain accuracy of the scheduling interval, the next run time of an Activity is computed at the start of the current execution. Chapter 5: Introduction to LEI Activities 63 This prevents variation in the amount of time it takes an Activity to complete from affecting its next scheduled run time. If an Activity does not complete until after the next scheduled run time, it will complete the current execution and then restart immediately to complete the overdue scheduled run. For example, an Activity has a repeat interval of 60 minutes and typically takes only few minutes to complete. The Activity will run on its hourly interval such as 6:30 PM, 7:30 PM, and so on. However, if the second execution takes more than 60 minutes to complete, the Activity will complete the 7:30 PM run after 8:30 PM. It will then start the 8:30 PM run as soon as the previous run is complete and, assuming it returned to its typical execution of only a few minutes, would be scheduled for 9:30 PM. When an Activity repeatedly overruns its next start time, the scheduling is adjusted to prevent continually scheduling overdue times. All of the scheduling settings are used to determine when the Activity will start. For many Activities, they start according to the schedule and then run to completion. The Polling and Advanced RealTime Activities also use the scheduling settings to determine when to stop. The Run at Times, Days of the Week, Weeks of the Month, and Days of the Month settings are used to determine valid start times. These same values are used to determine invalid times. For Activities which require a stop time, the first invalid time following the start time is computed and used as the stop time. Note If no values are specified for Run at Times, Days of the Month, and Weeks of the Month, and every weekday is listed in Days of the Week, then there are no invalid times and the Activity will not have a stop time. Also, the start time is computed before the stop time. As described previously, the start time is computed using the Repeat Interval. It is possible to have the next start time occur before the current stop time. For example, a Notes Virtual Fields Activity with valid run times being 7:00 AM - 3:00 PM and a repeat interval of 60 minutes would schedule to start at 7:00 AM and stop at 3:00 PM. However, it would have its next run time computed as 8:00 AM. When the Activity completed at 3:00 PM, it would view the 8:00 AM start time as overdue, and restart immediately. You can use the Restrict to Schedule setting to prevent this behavior. An alternative is to set the repeat interval to 1 day, which causes the run time to be 7:00 AM each day. Using Restrict to Schedule and setting the repeat interval to a small value has the benefit of restarting the Activity immediately in the event it stopped prior to the computed stop time. 64 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Scheduling The scheduling process allows you to set two kinds of controls: • The window of time during which the Activity can take place — This window can amount to an absolute start and stop date and time or to a regularly defined period (such as every weekday between 7:00 AM and 6:00 PM). • The Activity interval — The interval defines how often the Activity will be executed within the schedule window, which can be from every minute to every few months. An Activity executes as soon as the window opens and on schedule thereafter. For example, an Activity scheduled to execute once an hour between 7:30 AM and 5:00 PM will first execute at 7:30, then at 8:30, and every hour until its final execution at 4:30 PM. To avoid scheduling inconsistencies between servers, the LEI administrator supplies a standard time to the LEI server when it is started. As a result, all LEI servers supported by a single administrator operate on the same time. LEI considers Daylight Savings Time when necessary. Note If you want the Activity to take place when an event occurs rather than on schedule, make it the dependent Activity of a Polling Activity. See the chapter entitled “Polling Activity” for more information. The following fields in the Scheduling section of an Activity document enable you to define when and how often an Activity is executed: Basic Scheduling Field Description Run Activity Once and Disable Causes the Activity to run only once on schedule when checked. If scheduled, the Activity will run on schedule but then will not run again. After it is executed its Enabled/Disabled condition is changed to Schedule Disabled. Using Run ASAP to execute an Activity does not count towards its Run Once status. Run Once is useful for testing new Activities. continued Chapter 5: Introduction to LEI Activities 65 Field Description Schedule Specifies one of the three LEI scheduling options: Schedule Disabled, Schedule Enabled and Restrict to Schedule. With the cursor in the edit field, press the space bar to toggle between the three values. Schedule Disabled — Will not run based on any of the remaining scheduling information, but can still be run by clicking the Run ASAP button. Even though the Activity is currently “running ASAP,” the schedule status will still be shown as disabled. Shared execution also remains available. Schedule Enabled — Will run based on the remaining scheduling information. Restrict to Schedule — Similar to Schedule Enabled, but if the Activity execution starts outside a valid execution cycle, the Activity will not run and will be rescheduled for the next valid execution time. This option is used when an Activity should not execute with a delayed start, such as when the scheduled run time for an Activity passed while the LEI Server was down. When the server is started, overdue Enabled Activities start, but Activities marked Restrict to Schedule do not run until the next scheduled time. Activities are automatically disabled after a scheduled run if the Run Once option is checked. Repeat Interval Specifies the frequency of execution. The interval can be any number of minutes, days, weeks, or months. (Hours are entered as multiples of minutes.) Once an Activity is executed, its next run time is calculated by adding the interval to the last starting run time, as long as it falls within the window of execution defined in the Days of Week and Run at Times fields. If an interval of once a day or greater is used and no time of day window is defined, the next execution will be scheduled for midnight at the start of the next valid day. Run at Times Specifies the window of time during which the Activity executes. The delimiting times can be entered in AM/PM or 24-hour format and separated by a hyphen: 5:00 AM - 6:00 PM or 5:00 18:00 or separated by commas (the combination 4:00, 8:00 - 10:00 is valid). In either case, you must use the format hours:minutes (5 PM and 1700 are not valid). If no times are entered, the Activity can occur at any time. continued 66 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Days of the Week Specifies the days of the week on which the Activity will be executed. The Activity will be run during the Times of Day only on the days listed. To choose days, place the cursor in the day field and press Enter. Choose from the list. You can also add or delete directly: press the first letter of the day to insert it, making sure that days are separated by commas. Start Date and Time Optionally ensures that the Activity will not run before a specified date and time. Stop Date and Time Optionally ensures that the Activity will not be scheduled to run after a specified date and time. Days of the Month Ensures that the Activity will execute every month on the specified day(s). Weeks of the Month Specifies the weeks in the month to run the Activity. Use in combination with Days of the Month to finely control activity execution scheduling. Retry Options Enables you to specify a retry interval should the Activity fail prior to or during execution. Activity Retry on Error — When enabled, allows you to specify how many times to retry running the Activity at intervals specified in minutes. This option is useful in cases where the designated server may not be available to service the first request to run the Activity, so you would use this option to automatically keep retrying. Note When you use the Activity Retry On Error option, any retries that fall outside the specified schedule will not run. The base schedule will always dictate Activity run times, including the error retry attempts. Advanced Scheduling The Advanced Scheduling options enable you to schedule an Activity to execute on certain days or weeks of the month. It also allows you to specify a time in conjunction with a date. When both Days of the Month and Weeks of the Month are defined, both sets are valid. In other words, if weeks 1 and 2 and days 26, 27, and 28 are defined, then the Activity will be available for execution on days 1 through 14 plus 26, 27, and 28. The Activities, however, will always be limited to the Days of Week listed in the basic scheduling section. If the only listed day of the week is Monday, then the Activity will take place only when the days defined in the Advanced Scheduling section fall on a Monday. Chapter 5: Introduction to LEI Activities 67 Field Description Start/Stop Date and Time The dates and times that define the span of time during which the Activity may be scheduled to execute. You can set one or the other or both. A date and no time defaults to midnight. When just a time with no date is entered, the current date is used. If you intend to enter a start or stop date/time, however, it’s recommended that you enter both since the Administrator will not supply the missing information into the entry field. Enter a start or stop date/time only if you want to set a limit on the span of time that the Activity should be scheduled to start running. You can leave a date/time blank to set no start or stop limit on the Activity schedule. If you leave the start field blank, the Activity is first run when the document is completed and saved. An Activity that is running at Stop Time will continue to run until it completes, except for a Polling Activity, which, once started, runs up to Stop Time. If there is no stop time for the Polling Activity, it can still be stopped by using the Close command on the LEI server console. The date and time format to be used is specific to the international settings and the operating system on which the Notes Client is running. Days of the Month This option specifies on which days of the month the Activity can be executed. Day 1 is the first day of the month. The days should be listed separated by commas. Days entered as negative numbers count back from the last day of the month with -1 being the last day. If no days are listed, all days are valid (within the scope of other time-span definitions). For example: 7, 15, 21, -1 means that the Activity can be executed only on the seventh, fifteenth, twenty-first, and last day of the month. Range formats, such as 10 - 20, cannot be used. Each day in a range must be listed. Weeks of the This option specifies on which weeks of the month the Activity can be executed. The weeks should be listed separated by commas. Month Weeks entered as negative numbers count back with -1 being the last week (the last seven days of the month). If no weeks are listed, all weeks are valid (within the scope of other time-span definitions). Weeks are seven day intervals starting with the first day of the month, which may not necessarily be a Sunday to Saturday interval, and are defined as follows: w w w w w Week 1 — Days 1, 2, 3, 4, 5, 6, 7 Week 2 — Days 8, 9, 10, 11, 12, 13, 14 Week 3 — Days 15, 16, 17, 18, 19, 20, 21 Week 4 — Days 22, 23, 24, 25, 26, 27, 28 Week 5 — Days (29), (30), (31) 68 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Advanced RealTime Activities Advanced RealTime enables you to create the following three types of RealTime Activities. • Virtual Fields Activities (previously known as LEI RealTime or DECS RealTime) • Virtual Documents Activities • Virtual Agents Activities Virtual Fields Activity Virtual Fields Activity (previously known as LEI RealTime Activity or DECS RealTime Activity) provides access from a Domino application to a supported external data source, such as DB2. LEI, running on the Domino Server that is hosting the Notes application, intercepts and handles the Notes database events. For example, when Notes or Web client users open, create, update, or save Notes documents, these events are acted upon, obtaining immediate access from the Notes form to external data sources supported by Advanced RealTime. You get the data immediately (dependent, of course, on network bandwidth and other factors that affect system resources). Virtual Fields activity functionality includes features not available in the earlier RealTime Activity: support for computed subforms, options for new line delimiters, additional logging support, and support for procedure return parameters following insert and update operations. Output from write operations allows results of write operations to be returned to fields in the Notes document being monitored following insert and update operations. Once a system administrator has created the Virtual Fields Activity, Notes users can open, create, update or delete external, back-end data directly and transparently through their familiar Notes client. By extension, Web clients may open the same Notes forms by accessing a Domino R6 server, and obtain access to supported external source data. For example, if the external database to be queried or updated from the Notes form is DB2, Notes end-users may work with DB2 data as if it were in Notes. DB2 connectivity software is not required on the client system. Network access to the external data source is handled by the Domino server machine, which contains the connectivity software for the external data source, such as DB2. In addition, you have the option of storing retrieved data to the Notes form, or to simply view the retrieved data. Chapter 5: Introduction to LEI Activities 69 In a Virtual Fields activity, for each record in the external system, Notes contains a place holder document or “key document” which contains the database key value(s) needed to retrieve the data for that document. When the Domino application that uses the Virtual Fields Activity opens a document, the key value in the key document is used to retrieve the virtual fields from the back end. To the application user, the data looks as though it is stored in the Notes document. Virtual Documents Activity The functionality provided by a Virtual Documents activity is similar to that provided by a Virtual Fields Activity. The important distinction is that Virtual Documents do not require the presence of key documents in a Notes database. Instead Virtual Documents Activities store the NoteID (a Notes document unique identifier EIUNID and other pieces of information which determine the document’s unique identity) in the external system itself. These values serve as the unique keys to the external system records. As with all other Advanced RealTime functionality, a Virtual Documents activity is controlled by the LEI Administrator. You create a Virtual Documents activity document that specifies which Lotus Connector to use to access the external system and which Notes database and form to monitor. The connection document for the selected external system must be created before you create the activity document. When you start running a Virtual Documents activity, the activity monitors a Notes form for an event, such as opening or creating a document that uses that Notes form. For example, when you open the document the data that populates the form is retrieved from the external data source specified in the Virtual Documents activity document. When you create a new document, all data is stored in the external data source along with additional information that maintains the integrity of the Virtual Document for the next time the Virtual Document is accessed by Domino. For example, the Notes form information is preserved so the correct form is used in displaying the data the next time the Virtual Document is opened. Virtual Agents Activity A Virtual Agents Activity creates Domino Agents that run stored procedures on the external system database. The agents that it creates are added to the Domino Agent menu. If the stored procedures require parameters or if they return parameters, you must create Domino documents that hold these parameters; you then select the document or documents and run the agents against them. 70 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Integrated Credentials Database Like the Advanced RealTime Activities, the integrated credentials database is a recent addition to the LEI product offering. The integrated credentials database, available only for the three Advanced RealTime Activities, provides an additional and more robust security for associating a user ID and password to a specific external system database type, such as DB2, Sybase, or Oracle. You can build one or more integrated credentials databases, and name them using a meaningful naming convention, using the leicred.ntf template file supplied with LEI. In this database, you associate a user ID, password, and database type. The three Advanced RealTime Activity Documents enable you to specify that the credentials database is either used or not used for that specific activity name. The Activity Document choices are “Integrated Credentials: Use Connector Credentials”, in which case the credentials database is ignored, and “Integrated Credentials: Lookup Credentials”, which checks the named credentials database to ensure that the current user ID and password have authority to the external system specified on the Connection Document referenced on the active Activity Document. The Activity Document also provides a field in which you enter the file path to the credentials database. Lastly, you can also specify that, in the event there is a failure accessing the named credentials database, the credentials of the Activity Document be used instead. The Activity Document options that pertain to credentials checking are described in the three Advanced RealTime Activity chapters of this document — Virtual Fields, Virtual Documents, and Virtual Agents. Setting Up and Using an Integrated Credentials Database You use integrated credentials to gain access to an external system database. To set up an integrated credentials document you must create a Notes document based on the credentials template leicred.ntf supplied with LEI. You can name the resulting .nsf file anything you like. You can create and use a single integrated credentials database to contain all connection credentials for all of your external systems. You can optionally create multiple integrated credentials databases. Chapter 5: Introduction to LEI Activities 71 When you open the integrated credentials file, the following screen appears. The fields in the integrated credentials database are described below: • Allowed Readers — Specifies who has reader rights to this document. If a user is not listed, the document will not be visible to that user. • Notes Identifier — Specifies the Notes Username of the user requesting that the credentials be used. The Notes user ID is automatically obtained by LEI, entered in the field, and used as a search key. The ID value is derived from the Notes user ID editing the database. If adding the ID of another user or server, edit this field using the same format. For example, if the Notes user ID shown in this field is CN=John Doe/OU=Widgets/O=HugeCompany and you wish to add credentials for Bob Smith who works in customer support, edit the field to CN=Bob Smith/OU=Support/O=HugeCompany. The designations CN, OU, and O are built-in ID designations that correspond to the fully qualified Notes Domain name for a particular user or server. The designations stand for Common Name, Organizational Unit, and Organization. In some cases the OU designation may not apply. Consult your Domino system administrator if you are unclear of what to enter. • Connector — Specifies the applicable connector name (for example, “Sybase” or “DB2”). If this field is filled in, it associates the credentials to activities accessing this connector type (for example, to all Sybase connectors). This field is optional. • External Server — Specifies the external server name. This field is optional, however, if it is filled in, it restricts the credentials to activities accessing servers with this name. 72 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • External Database — Specifies the external database name. This field is optional, however, if it is filled in, it restricts the credentials to activities accessing databases with this name. • External Userid — Specifies the user ID to use for connecting to the external system. • Set Password — Specifies the password to use for connecting to the external system. The password will not be displayed in this form. Integrated Credentials and Full Text Indexing In order to perform full text indexing with Virtual Documents or Virtual Fields in conjunction with integrated credentials, you must add the server ID to the integrated credentials database. Integrated Credentials and Server to Server Replication In order to perform server to server replication in conjunction with integrated credentials, you must add the server ID of the server being replicated with to the integrated credentials database. For example, if you want to replicate a Virtual Documents or Virtual Fields-enabled database on Server A to Server B, you must add Server B’s server ID to the integrated credentials database on Server A. Integrated Credentials in a Domino Cluster The server IDs for all the servers in a Domino cluster that contain replicas of the Virtual Documents or Virtual Fields-enabled database must also be added to the integrated credential database. For example, if a Virtual Documents-enabled database on Server A has replicas on Servers B and C, and Servers A, B, and C are part of the same Domino cluster, the server IDs for Servers B and C must be added to the integrated credentials database on Server A. If this is not done, the cluster replication process will fail to keep the replicas synchronized. Integrated Credentials and Virtual Agents Virtual Agents only use integrated credentials for agent execution. Integrated Credentials and Mapping to a Stored Procedure When mapping to a stored procedure, the procedure name must be prefaced with the owner name in all instances where the owner name and the name by which the user has logged in differ. For example, if you are logged in as Emp54 and attempt to map to a stored procedure named abx owned by Emp98, you will receive a “Monitor failure error.”. To correctly map to that procedure, rename it to Emp98.abx. Chapter 5: Introduction to LEI Activities 73 Integrated Credentials in HTTP and Notes Clients The following considerations are helpful when using HTTP and/or Notes clients to access an LEI-enabled database in conjunction with integrated credentials. • If your Domino server is configured to allow anonymous access via HTTP, the user ID that will be looked up in the LEI credentials database will be “Anonymous.” In this case, to allow HTTP clients access you must a) have an entry for “Anonymous” in the LEI credentials database with valid database credentials, or b) have the “Use connector credentials if user credentials unavailable” option is checked. • If your Domino server is configured to deny anonymous HTTP access, and requires account information for HTTP access to the server, the user ID that will be looked up in the LEI credentials database for HTTP clients will be the “Short name” for the account that you logged into the Domino server with initially. This name is listed in the server’s address book entry for the user, on the “Basics” tab under “Short name/userID,” and is usually the first initial and last name of the user (such as “jsmith”). In this case you must have credentials entries for the short name in your LEI credentials database to allow HTTP client access. • When a user accesses the database from a Notes client, the user ID that will be looked up in the LEI credentials database will be the full Notes user ID for the user. In this case, you must have an entry for the full Notes user name (such as “John Smith/CAM/Lotus”) in the LEI credentials database to allow Notes client access. Note If you are using integrated credentials with Virtual Documents through an HTTP client, you also need a server credential document. Note To provide the same access through an HTTP client and through a Notes client to a particular user, you must have credentials records under both the user’s short name and the full Notes user name. 74 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Command Line Execution of Activities LEI allows activities to be executed from the system command line of the machine where an LEI server is running on a host. You might use this capability to execute LEIACT in response to specific database events. Command syntax for starting an LEI Activity from the UNIX command line is as follows: leiact "Activity Name" Command syntax for starting an LEI Activity from the Windows command line is as follows: nleiact "Activity Name" Command syntax for starting an LEI Activity from the iSeries command line is as follows: CALL QNOTES/LEIACT 'Activity Name' Note On iSeries, if you are authorized to run the QNOTES/LEIACT program, you can launch an LEI activity from the command line and it will run under that process. Enter (in single quoted string) the name of the LEI Activity to be executed. The LEI Activity will access DB2/400 data under the iSeries user profile registered in the DB2 connection and will access Notes data under the user identified in KeyFileName. If the user ID in KeyFileName (resident in the notes.ini file) is password-protected, you will be prompted for the password within your interactive iSeries process. In all cases, Activity Name is the name of the existing LEI Activity that you want to run. For example, the following NT command line entry would invoke the quoted activity name: nleiact "ABX Direct Transfer Activity" General Considerations When an activity is started from the system command line, it does not appear as running in the LEI Administrator or on the server console. However, logging still occurs. The LEI server only runs activities that were scheduled from the Administrator. Also, when an activity is running from an LEI server, it will not run simultaneously on that, or any other, server until it has completed its current execution. This is referred to as exclusive execution. In contrast, when an activity is started from the system command line, its execution may be shared. This means that the activity may be run more than once simultaneously. At first this may not seem very valuable, but there are special cases where this capability is important. Performing searches for a Chapter 5: Introduction to LEI Activities 75 Web application or generating reports are two examples where shared execution is useful. When running an activity from the command line, please note the following: • Any dependent activities listed in the Activity Document are not executed. • The Activity Document’s document link to the log database is not updated to point to the new log entry for that run. Additional iSeries Considerations LEIACT in the qnotes library is public *EXCLUDE and should remain so to secure activities from being launched by unauthorized users. Activity performance will be better when launched using the LEI server. You will see a log entry for all activities from a call to LEIACT, but the status of that execution will not be recorded in the LEI activity itself (under last run status). Data Mapping During a Direct Transfer or Replication Activity, LEI maps columns in the source and target databases in one of the following three ways: • According to Field Name • According to Field Position • According to User Definition Understanding how LEI performs field mapping is particularly useful when configuring your Replication and Direct Transfer Activities. Some Lotus Connectors have unique data mapping requirements. One such example is identity column support when using the Lotus Connector for DB2. Another example is stored procedures feature support for the Lotus Connector for DB2. Note For information about data types, see the “LEI and Data” appendix. For unique data mapping considerations that may apply to your specific Lotus Connector, see the companion document Lotus Connectors and Connectivity Guide. 76 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide According to Field Names Data is transferred from a field in the source database to a field of the same name in the target. If the Activity’s command statement is a SQL query, you can use column aliases to match the source field to the destination. For example, “SELECT empno employee” maps the empno field in the source to the employee field in the target. Note The actual expression depends on the database SQL syntax. According to Field Position Data in the first field of the source result set is copied into the first field of the target database, and so on. If the command statement in the Activity selects a subset of columns from the metadata, they will be treated as a simple incremental series starting with 1. The order in which the columns are listed in the selection statement is the order in which they will be transferred. The SQL statement “SELECT column6, column4, column7 FROM table” maps column 6 of the source to field 1 of the target, column 4 to field 2, and column 7 to field 3. According to User Definition You supply field names for both the source and target, and data fields are mapped in the order provided. For example, source fields listed as “A, B, C” and target fields listed as “X, Y, Z” result in source field A moving to target field X, B to Y, and C to Z. When Field Numbers Don’t Match Since Replication does not require the same number of columns in both databases, mismatching numbers of columns only affects Direct Transfer Activities. The following occurs with these Activities: • More columns in the source result set than in the target metadata — The transfer will fail and you’ll get an error message. • Fewer columns in the source result set than in the target metadata — The transfer will succeed and the “orphan” columns in the target will be filled with null values. Chapter 5: Introduction to LEI Activities 77 Stored Procedures and Output Parameters Output parameter support enables LEI to return the results of a Direct Transfer or Virtual Fields activity, made using target stored procedures or ERP calls, into the source database based on a condition in the target database. For example, you can write a stored procedure to perform an insert into a DB2 table and then generate a status or confirmation number. The Direct Transfer Activity, using the stored procedure as the target metadata, then transfers the data to the external system (source connection). The data that is returned as a result set of the transfer is then directed to the fields in the source document that have been specified in the Stored Procedure Output Mapping — Source field of the Direct Transfer Activity document. Note If the field names do not match exactly, you must explicitly map them in the field list. The Direct Transfer Activity is the only LEI Activity that supports this capability. See the chapter entitled “Direct Transfer Activity,” for more information about using this feature. Currently, the Lotus Connector for DB2 and the Lotus Connector for SAP (available for purchase separately) are the only Lotus Connector types that support stored procedures for output parameters. A working knowledge of how your Connector supports output parameters for stored procedures is required for using the LEI Direct Transfer Activity to create and use a target stored procedure. To learn more about how your specific Lotus Connector supports this functionality, see the specific Connector chapter in the companion manual Lotus Connectors and Connectivity Guide. Note If the source or target metadata in an LEI Activity is designated as a stored procedure, all declared parameters of that stored procedure must be accounted for in the Activity (mapped or listed for return). If any parameters declared in the stored procedure are missing from the Activity a message is returned. For example, DB2 will return a message “SQL1109N -The specified dll <procedure> could not be loaded”. DB2 catalogues stored procedures by name and number of parameters. 78 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Example: Notes to DB2 Direct Transfer Activity This example uses DB2 output parameter code in a direct transfer between a Notes database (source) and a DB2 database (target). Several values are obtained from the Notes database, such as employee name and hire date, and inserted into the SP_EMPdb2 table in the DB2 database. When the insert function is complete, the stored procedure, named SP2, generates a current timestamp and assigns it to DATEPROC. DATEPROC is designated as output so the values get passed back to the source (Notes) database. 1. Create the Notes Connection Document and the DB2 Connection Document. For this example, the connection names are LEITEST and DB2v71 respectively. 2. Create the DB2 stored procedure. In this example, the stored procedure name is SP2. It is called as the target metadata on Direct Transfer Activity Document. Chapter 5: Introduction to LEI Activities 79 3. Create and complete the top portion of the Direct Transfer Activity Document. In this example, the source and target connection names are specified from step 1 above. Also, the stored procedure named SP2 is specified in the Table Name field of the target connection. The form that will be updated by the stored procedure, in this example, SP_EMPdb2, is specified in the Form Name field of the source connection. The field mapping is done by name, as shown in the Field Mapping section of the Activity Document. 4. Complete the bottom portion of the Activity Document. a. Enable the Target Metadata is Stored Procedure option in the Target Data section. b. Enable the Accept Return Values into Source. c. Enter DATEPROC as the Source and Target field in the Stored Procedure Output Mapping section. 80 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 5. Run the Activity. Example output is shown below. In this example, ENAME, EMPNO, MANAGER, and HIREDATE are inputs to the target database and DATE/TIME PROCESSED is the output from the target database. Moving Text List Data over Different Platforms In Activities such as Advanced RealTime, Notes databases store all data in the same canonical format (little-endian byte ordering) on all platforms. Notes data is automatically converted by LEI to the host platform’s byte ordering format when read (for example, when the key field in the Notes form is accessed and used for record queries on the host back end system), and back to Notes canonical format when saved to a Notes form. (Composite data is an exception: it is always left in canonical format). System Planning Considerations: When moving data between Domino servers and between Notes and other database systems, there is normally not a problem. In the case, however, where data is moved from one Domino server to another through another DBMS using different LEI server platforms, multi-value types (text list, number list, datetime list) become corrupted. For example, you use an NT LEI server to move a text list from Notes to DB2. If the data is moved from DB2 back to Notes through a second LEI server running under Windows NT, there is no problem. But if the second LEI server runs on a UNIX platform, the text list data in the destination database will be invalid. Chapter 5: Introduction to LEI Activities 81 Activity Logging Logging information is available in all of the LEI activity views. The log lists the time and date when an activity started and ran. It also contains information about any errors that occurred during processing of the activity. To view the log for a specific activity, highlight that activity name in the LEI Administrator and click the Current Activity Execution Log option in the action bar. Example output is shown below. Note You can obtain the same output by clicking the View Log button in an active Activity Document. To view the log history for all recent activities use the Log option in the Log History section of the Navigator. 82 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Example output is shown below. The very bottom section of the LEI Administrator can also be expanded to display information about active servers and jobs. Chapter 5: Introduction to LEI Activities 83 Chapter 6 Admin Backup Activity This chapter provides information about the LEI Admin Backup Activity. Introduction to the Admin Backup Activity The Admin Backup Activity creates a backup copy of your LEI Administrator database (decsadm.nsf) and, optionally, the LEI Script Vault database (leivlt6.nsf). When to Use the Admin Backup Activity Use the Admin Backup Activity when you want to create a safe backup of the LEI Administrator and associated documents. You should use the Admin Backup Activity at regular intervals to ensure that you have backup copies of existing LEI activities, connections, and configurations. How to Create an Admin Backup Activity The steps below outline the general procedure for creating an Admin Backup Activity. See the other sections in this chapter for more information about the fields and options on the Admin Backup Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Admin Backup from the resultant list. 3. Complete the required fields and select desired options in the Admin Backup Activity document. 4. Save the Activity Document. 85 Admin Backup Activity Document The Admin Backup Activity Document is shown below. To create a new Admin Backup Activity, click Add Activity and then select Admin Backup from the resultant list. To open an existing Admin Backup Activity, simply double-click its name in the Activities view. Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options. 86 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Backup Options The backup options are described in the following table. Field Description Target Server Specifies the name of the Domino server where the backup copy of the LEI Administrator database and/or LEI Script Vault database will be stored. You must specify a valid target server. Target Database Specifies the file name of the backup copy (a Notes .nsf file) of your LEI Administrator. If the database does not exist, LEI will create it. Otherwise, the existing database will be overwritten. In addition, the file path can be assigned, for example, \qe\leiback.nsf. A copy of your LEI Administrator will be created with this name. The path name specified must be relative to the data directory. Leave this field blank if you do not want to back up the LEI Administrator. Script Vault Specifies the file name of the LEI Script Vault copy (as a Notes Target Database .nsf file). If the database does not exist, the Activity will create it. Otherwise, the existing database will be overwritten. In addition, the file path can be assigned, for example, \qe\leivlt6b.nsf. A copy of your leivlt6.nsf file will be created as this name. A copy of your LEI Script Vault will be created with this name. Note Only the scripts in the LEI Script Vault (leivlt6.nsf) that have been created as Shared Agents are copied during this Activity. This Activity does not copy private agents from leivlt6.nsf to the target database. To ensure that your scripts are copied to the target backup database, enable the Shared Agent option in Domino Designer when you create your LS LSX scripts. The path name specified must be relative to the data directory. Leave this field blank if you do not want to back up the LEI Script Vault. Note The LEI server must have manager access to the backup database in order to delete the previous backup. Since the backup’s ACL is copied from the Administrator database’s ACL, the server needs manager access to the LEI Administrator. Chapter 6: Admin Backup Activity 87 Chapter 7 Admin Purge Log Activity This chapter provides information about the LEI Admin Purge Log Activity. Introduction to the Admin Purge Log Activity Use the LEI Admin Purge Log Activity to purge the LEI log database of documents older than a user-specified number of days. When to Use the Admin Purge Log Activity You should use the Admin Purge Log Activity for the LEI logs at different times, depending on your situation. For example, if the LEI logs take up too much disk space, run this Activity regularly. This activity deletes existing LEI log documents. How to Create an Admin Purge Log Activity The steps below outline the general procedure for creating an Admin Purge Activity. See the other sections in this chapter for more information about the fields and options on the Admin Purge Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Admin Purge Log from the resultant list. 3. Fill in the required fields and select the desired options for the Activity in the Admin Purge Log Activity document. 4. Save the Activity Document. Saving Selected Logs before Purging To save specific LEI logs, you must copy them to another Notes database before running the Admin Purge Log Activity. 89 Admin Purge Log Activity Document The Admin Purge Log Activity Document is shown below. To create a new Admin Purge Log Activity, click Add Activity and then select Admin Purge Log from the resultant list. To open an existing Admin Purge Log Activity, simply double-click its name in the Activities view. Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. 90 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Purge Log Options The Purge Log Options Older Than value specifies the age for log records. All documents older than the specified number of days will be deleted from the log. The number of days are counted from the time and date that the Activity is run. Each day is considered a 24 hour period. Chapter 7: Admin Purge Log Activity 91 Chapter 8 Archive Activity This chapter provides information about the LEI Archive Activity. Introduction to the Archive Activity An Archive Activity moves data from one database to another. As records are moved into the target database, they are deleted from the source one record at a time. The source data and target location are indicated by metadata name (for example, table, form, and so on). Selection of documents to archive can be done with a condition or relative time stamp. The Archive Activity deletes the original records from the source database. Note When using an Archive Activity, you cannot use an ODBC Connection Document if you are accessing a SQL Server 7 database. To access the SQL Server 7 database, use an OLE DB Connection Document instead. When to Use the Archive Activity You can use the Archive Activity to protect data on a regular basis, for example: • To place infrequently accessed data to a removable storage device to free space on a system server • To protect data when migrating from one database to another • To create space on a full disk 93 Combining an Archive Activity with other Activities You can use the Archive Activity in combination with other LEI Activities. For example, you can combine the Archive Activity with a Polling Activity to Poll your system databases to determine when data was last accessed, and then archive it based on a specified date of its last access. How to Create an Archive Activity The steps below outline the general procedure for creating an Archive Activity. See the other sections in this chapter for more information about the particular fields and options on the Archive Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Archive from the resultant list. 3. Fill in the required fields and select the desired options for the Activity in the Archive Activity document. Note All connections referenced in the Activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. 94 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Archive Activity Document The Archive Activity Document is shown below. To create a new Archive Activity, click Add Activity and then select Archive from the resultant list. To open an existing Archive Activity, simply double-click its name in the Activities view. See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options, Scheduling, and Other options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Chapter 8: Archive Activity 95 Source Source options are described below. Field Description Edit Connection The Edit Connection button opens the named Source Connection Document in Edit mode. Connection Specifies the name of the Source Connection. Data is transferred from the Source Connection database to the Target Connection database. Each connection identifies the server and database with access information. The Form/Table/View Name option (see below) identifies the metadata. Form/Table/View Name Specifies the metadata, based on the type of connection chosen. For example, for a relational database the metadata is a Table, for a Notes database the metadata is a Form, and for a DB2 database the metadata is a view. Target Target options are described below. Field Description Edit Connection The Edit Connection button opens the named Target Connection Document in Edit mode. Connection Specifies the name of the Target Connection. Data is transferred from the Source Connection database to the Target Connection database. Each connection identifies the server and database with access information. The Form/Table/View Name option (see below) identifies the metadata. Note To create a table in the target, you can enter a name in the Target Table Name field and in the Archive Options section (see below), you can click “Create Target Metadata.” Form/Table/View Name Specifies the metadata, based on the type of connection chosen. For example, for a relational database the metadata is a Table, for a Notes database the metadata is a Form, and for a DB2 database the metadata is a view. 96 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Mapping The Mapping section of the Activity Document enables you to specify either manual or automatic field mapping. The default is manual mapping. Both mapping styles are described here. For related information, see the Data Mapping section in the “Introduction to LEI Activities” chapter. Manual Mapping The manual mapping fields are described below. Field Description Source Field(s) Provides space for you to enter the exact field names in the source to be mapped to names in the target. When you select a field in the source, the name automatically populates the target field list. You can either leave the field name in the target list, or move/edit/delete it in the target field list. To define your own field mappings, enter the names of the fields in the Source Field(s) and in the Target Columns in the proper mapping order. The first source field maps to the first target field, the second to the second, the third to the third, and so on. Target Columns Provides space for you to enter the exact field names in the target to be mapped to names in the source. Select Statement Contains the connection-specific select statement to be executed. Some sample syntax is supplied by default. Enter a command statement that identifies the data to be selected and transferred. The statement must conform to the syntax rules of the database engine (for example, a SQL query or Notes selection formula). A SQL query can be designed to select data from one or more metadata objects. Chapter 8: Archive Activity 97 Automatic Mapping When you select the Automatic option, the automatic mapping fields appear. They are described below. Field Description Automatic If you select Automatic, data is mapped from each source column to the destination by position (first column to first, second to second, third to third, and so on) or by field name. Selecting this option displays the following two options. w By Name — Automatically performs all mapping from the source to the target based on an exact name match between the two. w By Position — Automatically performs all mapping from the source to the target based on an exact position match (left to right, top to bottom) between the two. Select Statement Contains the connection-specific select statement to be executed. Some sample syntax is supplied by default. Enter a command statement that identifies the data to be selected and transferred. The statement must conform to the syntax rules of the database engine (for example, a SQL query or Notes selection formula). A SQL query can be designed to select data from one or more metadata objects. Archive Selection Options When you select the Archive Options tab and the Selection Options tab, the following options appear. 98 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Timestamp Archive When you enable the Timestamp Archiving option, the following Timestamp Field and Timestamp Cutoff in Days options appear. Timestamp Field Specifies the name of the field that contains the timestamp values that LEI will use to measure date and time from. Note If using a DB2 external system, the DB2 column selected for the Timestamp Field must be of type TIMESTAMP, not DATE. Timestamp Cutoff in Days Specifies the minimum age of records to be timestamped. For example, to specify that all records older than 30 days be archived, enter 30 in this field. Conditional Archive Optionally contains a conditional clause, written in the language of the source connection, to determine which records are to be archived. This allows you to use conditional criteria to choose specific record names in a table. Archive Precision Options When you select the Archive Options tab and the Precision Options tab, the following options appear. Field Description Generate Error for any Data Loss Specifies that LEI write an error to the log for any data that is lost as a result of the transfer and that it then terminate the transfer. Allow Precision Loss Only Specifies that LEI not report loss of numerical or datetime precision as a result of the transfer. Truncate Data When Necessary Specifies that LEI truncate text data when necessary to conform to field lengths in the target database. Because of performance penalties, this option does not affect numbers being written. Chapter 8: Archive Activity 99 Archive Target Data Options When you select the Archive Options tab and the Target Data Options tab, the following options appear. Field Description Existing Data Options Selecting Create Target Metadata enables LEI to create a new metadata in the database identified in the target to Create Target hold the data being transferred if one does not already exist. Metadata In creating new metadata, LEI attempts to match the source metadata as much as possible. If you do not select Create Target Metadata, LEI displays an error message if a specified target table does not exist. Note When archiving data from a Notes source with “Create Target Metadata” selected, Notes Text fields create character objects of size 64996 bytes (the maximum size of a Notes Text field) in some RDBMS external systems. 100 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 9 Command Activity This chapter provides information about the LEI Command Activity. Introduction to the Command Activity A Command Activity executes operating system, database, and SQL commands. In order to execute an operating system command, select No Connector and enter the command as it would be entered at an operating system command prompt. To execute a database or connection-specific command, select the appropriate Connection Document. When to Use a Command Activity You should use the Command Activity when you need to execute a specific command either on a connected database or on the operating system on which the LEI server is running. You can use the Command Activity to execute an SQL statement on a supported connection or execute an operating system command. How to Create a Command Activity The steps below outline the general procedure for creating a Command Activity. See the other sections in this chapter for more information about the particular fields and options on the Command Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Command from the resultant list. 3. Fill in the required fields and select the desired options for the Activity in the Command Activity Document. 101 Note All connections referenced in the Activity must exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. Command Activity Document The Command Activity Document is shown below. To create a new Command Activity, click Add Activity and then select Command from the resultant list. To open an existing Command Activity, simply double-click its name in the Activities view. 102 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options, Scheduling, and Other options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Connection The Connection Name field entry specifies the name of the connection. The command statement will be executed against this connection. The connection identifies the server and database along with required access information. To execute a command in the local operating system, specify no connection. The Edit Connection button opens the named Connection Document in Edit mode. Note When using no connection (for example when submitting an OS command), any command that takes input must have that input redirected from a file. If not, then this Command Activity will wait for input at the LEI console, and the Activity will seem to hang. For example, while the command “del *.*“ on NT will hang as it waits for confirmation, the command “del *.* < input.txt”, where input.txt contains a “y” followed by a new line, will process correctly because the confirmation has been redirected to come from the file rather than from user input (stdin). Command The Command Statement entry specifies the Connector-specific (or operating-system specific, when no Connector is selected) statement that identifies the action to be performed. The statement must conform to the syntax rules of the database engine or local machine (for example, a SQL query or Notes selection formula). Chapter 9: Command Activity 103 Command Activity Examples Following are some examples that illustrate how to use the Command Activity. Command Activity Using SQL This Command Activity inserts a row in a DB2 table. This example shows how you can manipulate data in a relational database using a SQL statement within a Command Activity. 104 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Command Activity Using a Notes Agent This example runs a Notes agent using the EXECUTE command statement. Notice that no quotes are required around the name of the Notes agent. The EXECUTE command statement is used to run a single Notes agent. No command parameters are used. The agent shown in the next figure, as it appears in Domino Designer, is executed by the above Command Activity. To open Domino Designer and an Agent view from within LEI, choose Create - Design - Agent from the Notes top bar. Note This particular agent is designed to run against “All documents in the database.” The only other valid document selection option for an agent is “Run Once.” Because agents cannot be view-based, the Domino Designer options “Run on selected documents in view” and “Run on all documents Chapter 9: Command Activity 105 in view” are not supported. See the “Lotus Connector for Notes” chapter in the companion Lotus Connectors and Connectivity Guide for more information about Activity Command statements. 106 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Command Example Issuing Operating System Commands This example executes an operating system-specific command. The Command Activity does not require that you specify a Connection Name. This allows you to enter command statements directed at the system. Another command statement example is shown below: del /q V:\SalesByMonth\June\*.* You do not need to specify a File connection to use this type of command statement. Note Inadvertently inserting a space in a file path can corrupt the Domino and LEI installations by deleting the notes.ini file. Chapter 9: Command Activity 107 Chapter 10 Direct Transfer Activity This chapter provides information about the LEI Direct Transfer Activity. Introduction to the Direct Transfer Activity A Direct Transfer copies data from one database to another. The data to be transferred is identified by a statement, for example a SQL query or Notes selection formula. You can start a Direct Transfer Activity either by clicking the Start button on an open Direct Transfer Activity Document, highlighting the Activity name in the Activity View of the Administrator and clicking Start, or by scheduling the Activity using LEI scheduling options. When to Use a Direct Transfer Activity You should use the Direct Transfer Activity when you want to move data from one database to another. You can also combine a Direct Transfer Activity with a Polling Activity in order to perform the data transfer when a specified condition is met. In some situations, you can enable output parameter support in order to update a field in the data source when the data transfer to the target is successful. How to Create a Direct Transfer Activity The steps below outline the general procedure for creating a Direct Transfer Activity. See the other sections in this chapter for more information about the particular fields and options on the Direct Transfer Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Direct Transfer from the resultant list. 109 3. Fill in the required fields and select the desired options for the Activity in the Direct Transfer Activity Document. Note All connections referenced in the Activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. There is an example of building a Direct Transfer Activity in the “Tutorials” appendix. Direct Transfer Activity Document The Direct Transfer Activity Document is shown below. To create a new Direct Transfer Activity, click Add Activity and then select Direct Transfer from the resultant list. To open an existing Direct Transfer Activity, simply double-click its name in the Activities view. Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution, Scheduling, and Other options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. 110 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Source Source options are described below. Field Description Edit Connection The Edit Connection button opens the named Source Connection Document in Edit mode. Connection Name Specifies the name of the Source Connection. Data is transferred from the Source Connection database to the Target Connection database. Each connection identifies the server and database with access information. The Form/Table/View Name option (see below) identifies the metadata. Form/Table/View Name Specifies the metadata, based on the type of connection chosen. For example, for a relational database the metadata is a table, for a Notes database the metadata is a form, and for a DB2 database the metadata is a view. Target Target options are described below. Field Description Edit Connection The Edit Connection button opens the named Target Connection Document in Edit mode. Connection Name Specifies the name of the Target Connection. Data is transferred from the Source Connection database to the Target Connection database. Each connection identifies the server and database with access information. The Form/Table/View Name option (see below) identifies the metadata. Form/Table/View Name Specifies the metadata, based on the type of connection chosen. For example, for a relational database the metadata is a table, for a Notes database the metadata is a form, and for a DB2 database the metadata is a view. Chapter 10: Direct Transfer Activity 111 Mapping The Mapping section of the Activity Document enables you to specify either manual or automatic field mapping. The default is manual mapping. Both mapping styles are described here. For related information, see the Data Mapping section in the “Introduction to LEI Activities” chapter. Manual Mapping The manual mapping fields are described below. Field Description Source Field(s) Provides space for you to enter the exact field names in the source to be mapped to names in the target. When you select a field in the source, the name automatically populates the target field list. You can either leave the field name in the target list, or move/edit/delete it in the target field list. To define your own field mappings, enter the names of the fields in the Source Field(s) and in the Target Columns in the proper mapping order. The first source field maps to the first target field, the second to the second, the third to the third, and so on. Target Columns Provides space for you to enter the exact field names in the target to be mapped to names in the source. Select Statement Contains the connection-specific select statement to be executed. Some sample syntax is supplied in the pop-up help. Enter a command statement that identifies the data to be selected and transferred. The statement must conform to the syntax rules of the database engine (for example, a SQL query or Notes selection formula). A SQL query can be designed to select data from one or more metadata objects. Automatic Mapping When you select the Automatic option, the automatic mapping fields appear. They are described below. Note If you select to “Create Target Metadata” and you have a Notes/Domino database as a source and you have a RDBMS external database, such as DB2, as the target, the Notes Text fields create character objects of size 64996 bytes in the target database. To limit the size of text fields, edit the Notes Connection document option “Maximum Length for Text Data.” 112 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Automatic If you select Automatic, data is mapped from each source column to the destination by position (first column to first, second to second, third to third, and so on) or by field name. Selecting this option displays the following two options. w By Name — Automatically performs all mapping from the source to the target based on an exact name match between the two. w By Position — Automatically performs all mapping from the source to the target based on an exact position match (left to right, top to bottom) between the two. Select Statement Contains the connection-specific select statement to be executed. Some sample syntax is supplied in the pop-up help. Enter a command statement that identifies the data to be selected and transferred. The statement must conform to the syntax rules of the database engine (for example, a SQL query or Notes selection formula). A SQL query can be designed to select data from one or more metadata objects. Direct Transfer Options The Direct Transfer Options provide selections for data handling during the Activity. For example, enabling output parameters for stored procedures is accomplished using this portion of the Activity Document. Each of the available options are described in the following tables. Precision Options When you select the Direct Transfer Options tab and the Precision Options tab, the following options appear. Chapter 10: Direct Transfer Activity 113 Field Description Generate Error for any Data Loss Specifies that LEI write an error to the log for any data that is lost as a result of the transfer and that it then terminate the transfer. Allow Precision Loss Only Specifies that LEI not report loss of numerical or datetime precision as a result of the transfer. Truncate Data when Necessary Specifies that LEI truncate text data when necessary to conform to field lengths in the target database. Because of performance penalties, this option does not affect numbers being written. Source Data Options When you select the Direct Transfer Options tab and the Source Data Options tab, the following options appear. Field Description Source Data Selecting the Source Metadata is Stored Procedure option enables you to call a stored procedure from the source database as a means of obtaining a result set. This feature is available only for Connectors that support stored procedures that can return result sets or output parameters. The stored procedure can never have input parameters declared. The Select Statement will not be used when this option is enabled. 114 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Target Data Options When you select the Direct Transfer Options tab and the Target Data Options tab, the following options appear. Field Description Existing Data Options: Create Target Metadata LEI will create a new metadata in the database identified in the Target Connection to hold the data being transferred if one does not already exist. In creating a new metadata, LEI will try to match the source metadata as much as possible. If the source table contains no records, the Direct Transfer Activity will not transfer the table. To transfer the table, in cases where all table records contain NULL entries, add one record to the table and then perform the Direct Transfer Activity. Note When transferring from a Notes source with “Create Target Metadata” selected, Notes Text fields create character objects of size 64996 bytes (the maximum size of a Notes Text field) in some RDBMS external systems. To limit the size of text fields, edit the Notes Connection document option “Maximum Length for Text Data.” Existing Data Options: Overwrite Existing Data Checking this box causes all the data in the destination metadata to be deleted before the new data is transferred. Note that even if the transfer fails to complete successfully, the data already has been deleted from the target and will be lost. If this option is not checked, the transferred data is appended to the existing data in the target. Insert Options: Try When selected, this option prompts for a key field to use to Update Before Insert check for records to update. continued Chapter 10: Direct Transfer Activity 115 Field Description Insert Options: Target Key Fields for Update The target key field is used to find the data record in the target, and, if found, the record will be updated (whether it changed or not) on the destination. If a record from the source (based on key field) does not match with a record in the target, then that record will be inserted into the target as a new record. Options: Target Metadata is Stored Procedure When set, the target metadata indicates a stored procedure to execute. The fields being transferred become input parameters to the stored procedure with the field names as parameter names. If selected, the Accept Output Parameters into Source menu option appears. Options: Continue on Insert Errors Enable this option to have the activity log all insertion errors and continue with the transfer. Options: Disable Trimming of Text Trailing Spaces Specifies that trailing spaces in a text field not be trimmed. The default behavior is to trim trailing spaces. Accept Output Parameters into Source If you have enabled “Target Metadata is Stored Procedure”, this option appears. This option enables the return of parameters from the stored procedure to fields in the source connection. For DB2 users, enabling this option opens a user-definable field mapping section. Use this field mapping section to map fields in the source connection (metadata) to target fields designated as output parameters declared in the stored procedure. For SAP users, enabling this option opens a user-definable field mapping section. Use this field mapping section to map fields in the source connection (metadata) to either target fields designated as export parameters or to the result set returned by the stored procedure. Note Each output parameter fetched affects the activity’s logged fetch count. The fetch count value is the total number of records fetched for the overall activity. Since the outputs for each procedure call needed to be fetched, the activity log will display twice as many fetches as expected: half from the source and half from the target. This is the correct and intended behavior. You can map the source fields with the stored procedure output parameters in the Stored Procedure Output Mapping section. continued 116 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Stored Procedure Output Mapping: Source Fields Lists the fields in the source connection that will receive the stored procedure output. The first source field listed will map to the first output parameter listed and so on. Stored Procedure Output Mapping: Target Parameters Lists parameters in the target connection that will return data to the source. The first parameter listed will map to the first source field listed and so on. Performance Options When you select the Direct Transfer Options tab and the Performance Options tab, the following options appear. Field Description Number of Records to Transfer Concurrently Enables buffering of records prior to transfer. By default, one record is transferred at a time. For databases and connections that support array transfers, this number defines the number of records to transfer at a time. Transferring multiple records at a time can increase performance, but only with databases that support the feature. Note If the Target Metadata is a Stored Procedure option is enabled, the only valid entry in this field is 1. Maximum Number of Records to Transfer Specifies the total number of records that you want this Activity to transfer. Zero (0) equals no limit. Chapter 10: Direct Transfer Activity 117 Direct Transfer Activity Considerations This section describes considerations for using the Direct Transfer Activity. Using Stored Procedures and Output Parameters Considerations for using stored procedures and output parameters are presented below. • Currently, the Lotus Connector for DB2 and the Lotus Connector for SAP (available for purchase separately) are the only Lotus Connector types that support stored procedures for output parameters. • If the source or target metadata in an LEI Activity is designated as a stored procedure, all declared parameters of that stored procedure must be accounted for in the Activity (mapped or listed for return). If any parameters declared in the stored procedure are missing from the Activity, a message will be returned. For example, DB2 will return a “SQL1109N - The specified dll <procedure> could not be loaded” (DB2 catalogues stored procedures by name and number of parameters.) See the “Lotus Connector for DB2” chapter of the Lotus Connectors and Connectivity Guide for more information. • When the Source or Target is a stored procedure, the Performance Option “Number of Records to Transfer Concurrently” only supports the default of one record. • Fields and stored procedure parameters are internally associated by name, so fields and parameters should always be explicitly mapped in the appropriate mapping sections. When using a stored procedure with output parameters as the target of a Direct Transfer, the Automatic Field Mapping By Position option should not be used. For an example of how to use stored procedures for output parameters, see the “Introduction to LEI Activities” chapter of the IBM Lotus Enterprise Integrator Activities and User Guide. Using the Lotus Connector for Notes When transferring from a Notes source with the Data Option “Create Target Metadata” selected, Notes Text fields create character objects of size 64996 bytes (the maximum size of a Notes Text field) in some RDBMS external systems (DB2 for example). To limit the size of text fields, edit the Notes Connection Document option “Maximum Length for Text Data.” 118 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Using the Lotus Connector for File System • If the Source or Target Connection of a Direct Transfer is a File connection and Automatic Field Mapping by Position is selected, the positional order of the File fields is as follows: Filename Contents Timestamp Size • If the Source Connection of a Direct Transfer is a File Connection, you can leave the select statement blank. The activity will proceed without an error as though the select statement had read as follows: select *.* Using Oracle or ODBC to Create Target Metadata In order to use a Direct Transfer Activity to create metadata that contains more than one column of type LONG VARCHAR (ODBC type that maps to Oracle’s LONG) you must use the Lotus Connector for Oracle 7, instead of the Lotus Connector for ODBC. Oracle does not support more than one column of type LONG in a table or in an SQL statement. Chapter 10: Direct Transfer Activity 119 Chapter 11 Polling Activity This chapter provides information about the LEI Polling Activity. Introduction to the Polling Activity Polling is a way of monitoring conditions in a database and triggering a subordinate Activity(s) when those conditions are met. As soon as the condition is met, the subordinate Activity(s) are executed. Subordinate Activities are defined in their own Activity document and are called by the Polling Activity. The connection used by the Polling Activity must remain available for as long as the activity is to run. LEI will make one, single attempt to reestablish a closed connection. If this attempt fails, the Activity will end in error. The Polling Activity does not reestablish a closed connection to a data source. For example, if you have configured Sybase to time out its connection after 10 minutes of inactivity, but your Polling Activity has a polling interval set to 15 minutes, the Sybase connection may time out and close before the Polling Activity checks the connection. in this case, the Polling Activity would fail. When to Use the Polling Activity You should use the Polling Activity when you need to execute some action when another action or event occurs or is encountered. For example, your Human Resources department may use an Oracle database for employee information. You may need to maintain this same information in a Notes database so that Sales and Marketing departments can access it. You could use a Polling Activity to check the Oracle database for new or changed records. When the Polling Activity detects such an event, it would trigger a Replication Activity to copy the changed data to the Notes database. Note The Polling Activity can run more than one activity at a time (for example, a Replication Activity and a Direct Transfer Activity). However, when you start the Polling Activity, the specified (subordinate) activities are not run in the sequence in which they are specified. If you want to run the specified activities in a given sequence, use the “Dependent Activities” field 121 in the General Options section of the activity document, see the chapter entitled “Introduction to LEI Activities” for more information about the General Options. How to Create a Polling Activity The Polling Activity Document contains the information needed to execute subordinate activities, including: • Condition under which the subordinate Activities (the Activities to execute) will be run • Scheduling and polling frequency • Names of subordinate activities to be executed Note Information about how to access the polled database must be defined in a Connection Document before the Polling Activity is created. All subordinate Activities must also be defined (in their own Activity Documents) before you can create a Polling Activity. The steps below outline the general procedure for creating a Polling Activity. See the other sections in this chapter for more information about the particular fields and options on the Polling Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Polling from the resultant list. 3. Fill in the required fields and select the desired options for the Activity in the Polling Activity Document. Note All connections referenced in the Activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. There is a complete example of building a Polling Activity in the “Tutorials” appendix. 122 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Polling Activity Document The Polling Activity Document is shown below. To create a new Polling Activity, click Add Activity and then select Polling from the resultant list. To open an existing Polling Activity, simply double-click its name in the Activities view. You can either request that polling occur based on a trigger statement or that polling occur based on a timestamp field; the two polling types are mutually exclusive. To request trigger-based polling, specify an appropriate value in the Table and Trigger Statement fields. To request timestampbased polling, enable the Timestamp Polling option and then specify an appropriate Timestamp field value. Chapter 11: Polling Activity 123 Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options, Scheduling, and Other options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Connection This section provides information on the source of the data to be polled. Connections and subordinate Activities must be created before they can be used in a Polling Activity Document. Field Description Edit Connection Opens the specified Connection Document (using the Connection Name below) in Edit mode. The Edit Connection button is only available after you specify a Connection Name. Connection Name Specifies the name of the connection containing the database to be polled. When you specify a Connection Name, the Edit Connection button appears. continued 124 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Table Specifies the metadata for trigger-based polling, based on the type of connection chosen. For example, in a relational database the metadata is a table, in a Notes database the metadata is a form, and in a file the metadata is a subdirectory. Trigger Statement Specifies the connection-specific trigger statement that you want to execute. A sample trigger statement is shown below: select * from SYM_TEST_SYBASE In this case, the trigger will be TRUE if any rows exist in the table. The trigger statement identifies the condition that will cause the Polling Activity to place subordinate activities in a queue for execution. The trigger statement is satisfied when it produces a result set containing at least one record. This is considered a successful poll. The trigger statement is only used when you request trigger-based polling. It is not used when you request timestamp-based polling. Timestamp Polling Check the Enable Timestamp Polling option to request timestamp-based polling and display the Timestamp Field and the Reset Timestamp options. The initial timestamp polling value is set to “01/01/0001 12:00:00 AM”, causing all activities specified for polling to run when the activity is started. Timestamp Field Specifies a data field that contains the datetime of interest. The Polling Activity will poll this datetime field for a value between the last poll datetime and the current datetime. If records fit this criteria, the Activities chosen in “Activities to Execute” will run. This field only appears when you select Enable Timestamp Polling. The timestamp field value is only used when you request timestamp-based polling. It is not used when you request trigger-based polling. Note If using a DB2 external system, the DB2 column selected for the Timestamp Field must be of type TIMESTAMP, not DATE. Reset Timestamp Enabling this option clears the stored timestamp value for this polling activity, causing all activities specified for polling to run the next time that this activity is run. A confirmation screen appears enabling you to choose Yes to Continue or No to Cancel. Chapter 11: Polling Activity 125 Definition When you select the Definition tab, the following options appear. Field Description Polling Frequency Specifies, in number of seconds, how often the database is to be polled to check for the condition defined in the Trigger Statement. The polling only occurs within the window of time defined in the Scheduling section of this Activity Document. See the “Notes on Polling Activity Scheduling” section later in this chapter. Activities to Execute Specifies the list of subordinate Activities to execute when the result of the Trigger Statement is true. The activities are listed alphabetically for convenience and do not reflect any execution sequence. To control the dependence of one activity on the completion of another activity, use the “Dependent Activity(s)” field in the General Options section of each activity. The Polling activity will run the specified activities independently of the activities’ schedules. Specifying an Activity here will have no effect on its own scheduling or whether it can be run using the Start! button. Note You do not need to enable Scheduling on the subordinate Activity Document in order for that activity to be run from the Polling Activity. continued 126 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Synchronous Setting Enabling the Execute Synchronously setting suspends polling while subordinate activities are executed. By default, this option is not checked and polling is not suspended while subordinate activities are executed. When polling is suspended while the subordinate activities are executed, polling will not resume if a subordinate activity cannot complete for any reason (for example, the server that was supposed to run it is not up). In this case, the Polling Activity will remain suspended until closed manually. Note If you enable both Execute Synchronously AND Reset Trigger Execution: Execute after Activities, the trigger will reset after the activity completes. Reset Trigger Statement Specifies an optional statement for resetting a trigger condition after a successful poll has occurred. It specifies the optional statement to execute after a successful poll. This statement can be used to reset the condition that caused the poll to succeed. For Notes, the statement must be a formula agent using the following formula: Execute agentname In Domino Designer, select Formula when you create the agent. You must also select the “All documents in database” option. When you enter a value in the Reset Trigger Statement field, the Reset Trigger Execution and Commit Options fields appear. Reset Trigger Execution Specifies when to reset: either before or after the activity has started. You can also choose to have the statement execute after the subordinate activity has completed — see Note below. w Execute before Activities — Executes the Reset Trigger before the subordinate activities are started. w Execute after Activities — Executes the Reset Trigger after the subordinate activities are started. Note If you also enable Execute Synchronously, then the trigger will reset the activity after the activity completes. If you do not also enable Execute Synchronously, then the trigger will only reset after the activity starts, not after the activity completes. Commit Options Commit Post Statement — For connections that support a Commit Action, this commits the Reset Trigger statement following each execution. Chapter 11: Polling Activity 127 Error Handing Options When you select the Error Handing Options tab, the following options appear. Field Description On Polling Error Ignore Polling Errors — Check this option to continue polling if the Trigger Statement returns an error (other than a statement that the connection has been lost). Maximum Event Count Specifies the number of times the Activity will poll successfully before terminating. A value of 0 specifies that there is no limit to the number of times polling can occur. Considerations for Polling Activity Scheduling The following considerations provide practical insight into scheduling Polling Activities. • A Polling Activity is an ongoing activity, unlike batch activities such as Direct Transfer or Replication. • Polling Activities should always be scheduled with Start and Stop times. If you use the Start and Stop time specifications, you do not need to enable the Restrict to Schedule option. • Polling Activities are only scheduled when they are run from the LEI Administrator. • If a Polling Activity is executing a poll at the exact Stop time, it will complete that poll. If that poll is successful, the Polling Activity will then run the subordinate activities. • Polling Activities may have dependent Activities as well as subordinate Activities. Dependent Activities are only run once the Polling Activity ends in one of the following ways: • The schedule expires • The Polling Activity is shut down by the user • The specified number of successful polls has been made 128 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • The Maximum Duration setting is designed to halt a single Activity that has been executing for too long and that may actually be hung. To specify that an Activity poll for a specific duration, such as 45 minutes or from 6:00 to 6:45, use Start and Stop time specifications on the Activity document. • When a Polling Activity is started, it makes a persistent connection to the external system which is not terminated until the Activity is closed by either the user or the schedule. • Even though the Polling Activity is typically only reading an external data source, the Read transactions must be committed. A Read is not committed by default. This may mean that other processes or users may be denied access to the table until the Read is committed. You should issue a Commit after every transaction in the Connection Document, including Read transactions. Example: Scheduled Polling Activity In this example, a user wants to do the following: • Poll every 60 seconds from 6:00 PM to 6:45 PM. • After one successful poll, execute another Activity. To accomplish this objective, set the following values on the Polling Activity document: • Run at Times: 6:00 PM - 6:45 PM • Maximum Duration [ ] • Repeat Interval [60] seconds Chapter 11: Polling Activity 129 Chapter 12 Virtual Fields Activity There are three Advanced RealTime Activities in LEI. • Virtual Fields (previously known as LEI RealTime or DECS RealTime) • Virtual Documents • Virtual Agents This chapter provides information about the Virtual Fields Activity. Introduction to Virtual Fields Activities LEI activities can monitor Notes forms. LEI connects these forms to external data sources (such as DB2) through Lotus connectors. Once a system administrator has created a Virtual Fields activity (previously known as the LEI RealTime Activity), you can open, create, update or delete external system data directly and transparently through their Notes client. By extension, Web clients can open the same Notes form that the activity monitors by accessing a Domino server and obtain access to supported external source data. For example, if the external database to be queried or updated from the Notes form is a DB2 database, Notes end-users may work with DB2 data as if it were in Notes. DB2 connectivity software is not required on the client system, however it must be installed on the Domino server machine. Network access to the external data source is handled by the Domino server machine, which contains LEI connectivity software for the external data source, such as DB2. Virtual Fields functionality includes the following features not available in the earlier RealTime activity: • Support for computed subforms, options for new line delimiters • Integrated credentials • Support for procedure return parameters following insert and update operations Output from write operations allows results of write operations to be returned to fields in the Notes document being monitored following insert 131 and update operations. One activity can monitor all forms in a database, making it possible to perform a generic lookup of a given field. Overview In Virtual Fields activities, Notes documents still exist for every data set presented to the user. Virtual Fields activities simply add external system data to an existing document. In the simplest Virtual Field scenario, the Notes document contains only the key value necessary to retrieve the external system data. These Notes documents are referred to as “key” documents (in previous documentation these were referred to as “stub” documents). While the Notes document can be a simple key document, it can also contain additional fields. Any field not mapped to external system data is saved in the Notes document. Each Virtual Fields activity monitors a specific Notes form where the metadata is defined. Metadata is the list of fields in the Notes form that are mapped to a list of fields in the external data source during data query or update. You must map the key fields in the external data source to the corresponding fields in the Notes form using the Virtual Fields activity. The Virtual Fields activity requires key documents (referred to as “stub documents” in previous releases) in the Notes database to function. These key documents must be initialized once and only once in the Virtual Fields activity. See below, “How Key Documents Work,” for more information. In addition, Virtual Fields activities can invoke external system stored procedures to produce the data to display in the form. The use of Virtual Fields in combination with Virtual Documents is documented in the “Using Virtual Fields with Virtual Documents” appendix. Multiple Virtual Fields Activities and Monitor Order Virtual Fields activities operate at the field level. You can have multiple Virtual Fields activities monitoring the same form in a Notes document, but each activity is responsible for a subset of the fields on a form. For example, if any of the Virtual Fields activities is not running, the document form contains all the fields, but only the fields for the stopped Virtual Fields activities will be blank. In brief, you can use one or more Virtual Fields activities to monitor a form in a Notes document, as long as you make proper use of monitor order where appropriate. Using monitor order you can have one activity that uses, for example, empid as a key to retrieve employee data, including the job ID. The job ID, could in turn be used as a key in another activity to retrieve the job title, description, salary range, and so on. In short, you do a lookup that gets a key to do the next lookup, and so on. 132 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide In summary, one or multiple external sources may be accessed from the Notes form. A single external data source definition (connection document) indicates the data source to connect to and the metadata to use. Many Virtual Field activities can be used to monitor a single form; each of these access data from multiple external data sources. The ability of Virtual Fields to aggregate information from multiple tables onto a single Notes document form is one reason why you might want to use a Virtual Fields activity rather than a Virtual Documents activity. In addition, you have the option of storing retrieved data to the Notes form, or you can simply view the retrieved data. See Data Storage under “General Options” later in this document. How Key Documents Work When you initially create a Virtual Fields activity for a database which has existing records, you use the “Initialize Keys” option to populate the Notes database with one “key document” for each external system record. These LEI initialized “key documents” contain the key value(s) that are mapped to the external system table and that you specified in the activity document. These key(s) are used to do the lookup into the external system table. When the Notes document is opened, the key(s) in the “key documents” are used to fetch external system data to populate the rest of the fields. Initializing Keys is a onetime task that is used to populate a new Notes database with the necessary key documents. Once you have initialized keys, the way to incorporate additional data that is added to the external source from outside of Notes and Domino is to use a Replication activity. When you create a new document through a Notes client, a “key document” is created and saved with the key field value(s). In the simplest case, all of the data resides in an external database and the Notes document only holds the lookup key, the note is really a place holder, or key document. However, the “key document” can be much more than just a container to hold the key(s). It can be used to contain other data in addition to the data stored in the external system. Any fields on the form which are not mapped to an external system field in the activity document will be stored in the “key document.” In addition, in the Virtual Fields activity document, mapped fields can be specified as fields that should be saved in the “key document”. In any case, all mapped field values are stored in the external data source. For example, you can construct a form, monitored by a Virtual Fields activity, that displays, employee history, payroll data, and personal information. In this situation, assume that the employee data and personal information is stored in a Notes database (nsf), and the payroll department information is stored in a DB2 database. In this case, the “key document” is only a “key” in the sense that the employee ID, for example, is used to fetch Chapter 12: Virtual Fields Activity 133 the payroll information when the employee’s information document is opened. If the Virtual Field activity were not running, or if the user did not have privileges to the DB2 data, the user would still see all of the information about the employee, but the payroll fields would be blank. However, if you had selected “Leave all real-time fields in documents” in the Data Storage section of the General Options, the data would retain the last refresh value before the Virtual Fields activity had stopped running. See “General Options” in this chapter for more information. Virtual Fields and Public Key Encryption If any virtual fields have been enabled for encryption, and a Public Encryption Key is provided in the document properties security tab, any and all encryption enabled fields will be encrypted as with non-virtual fields. The corresponding data in the external system table will be nulled. The encrypted data will only be visible to Notes users with the appropriate encryption key. External system table columns which are mapped to encryption enabled virtual Notes fields should not have constraints preventing NULL values or uniqueness constraints. Any encrypted virtual field data will be stored within the key (stub) document. Usage Requirements, Performance, and Mapping in Virtual Fields Activities There are three usage requirements to keep in mind: • Indexing • Key Field Datatypes • HTTP Server The indexing requirement is necessary for proper Virtual Fields activity operation, as well as key field datatypes the HTTP server. Indexing For Virtual Fields, the following fields should be indexed: • Mapped key fields • All Sybase data tables 134 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Key Field Datatypes Any field used for a key field in a Virtual Fields activity must be usable as a key field in the external system data source. For each of the data sources listed below, the specified data types cannot be used as key fields in Virtual Fields activity. • Oracle — LONG and LONG RAW types cannot be used as keys. • DB2 — BLOB, CLOB, and DBCLOB types cannot be used as keys. • Sybase — TEXT and IMAGE types cannot be used as keys. • ODBC — Varies by specific external system; refer to the database documentation. • Notes — RICH TEXT fields cannot be used as keys. Note Refer to the data source documentation for more information about the use of data types as key fields. HTTP Server When using the HTTP server to access documents, do not use the Virtual Fields update event option “Conflict Detection” because it relies on the use of hidden fields and the HTTP server does not save context information. Performance in Virtual Fields To maximize performance in Virtual Fields activities, when the external connection is an SQL database, make sure an SQL index exists on the key fields. For example, if the key fields for a Virtual Fields activity monitoring table “MyTable” are “FirstName” and “LastName,” create an index in the SQL database as follows: CREATE UNIQUE INDEX IndexName on MyTable (FirstName, LastName) When the Virtual Fields activity is enabled, LEI opens the connection when the first event for that activity is processed. LEI closes the connection as soon as possible after the activity has been scheduled to stop. By default, a single persistent connection is shared by all users of the Notes database. It is possible to set the maximum number of concurrent, opened connections to increase performance. See the Max. Concurrent Connections options in the Virtual Fields Options section. Mapping and Unmapped Columns In general, if there are unmapped columns in the Virtual Fields table, which do not accept NULLs, you will get a NULL assignment error when creating a new record. The exact wording of this error depends on the connector you are using. This situation is particularly noticeable when you use a Virtual Documents table as a “key document” for a Virtual Fields activity. Chapter 12: Virtual Fields Activity 135 Creating a Virtual Fields Activity Using the User Assistant You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Fields Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. This section describes how to create a Virtual Fields Activity using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” Step 1: Open the Activity Document In either the Activity View or the Favorites View (or a Folder View), click the Add Activity icon in the Action bar. A message box opens that explains the steps the User Assistant will guide you through. 136 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Note If you have not made any selections while the User Assistant is on, clicking the Cancel button on this message box closes the activity form and returns you to the LEI Administrator. Once you have made one or more selections while you are in the User Assistant, clicking Cancel causes the Assistant to shut down, but the activity document stays open, in Edit mode, displaying the choices you have made up to that point. You can then manually complete the activity document or abandon it. Any choices you make while using the Assistant can be changed later by opening the activity document in Edit mode. Step 2: Select the Domino Database A list of Domino databases appears (see illustration below). Select the Domino database that contains the Notes form you want to monitor. Note When the list is large, you can scroll down the list to find the domino application (nsf) you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Chapter 12: Virtual Fields Activity 137 Note Immediate or “real-time” monitoring uses the Notes replication ID of the application database. A Virtual Fields activity does not distinguish between the databases that share the same replication ID. Databases have the same replication ID when they are replicas of each other and when one database is created using the operating system to copy an existing database. Step 3: Select the Notes Form to Monitor A list of forms within the selected database appears. These are the forms you will use to display external system data. Select a single form that you want to monitor from the list shown. Note Use the “Form Override” check box in the “General Options” section of the Virtual Fields Activity document to indicate that the Virtual Fields Activity should monitor all forms within the database, regardless of the form it uses. This option is only applicable to Virtual Fields Activity documents. See the section on “Virtual Fields Activity Options” later in this chapter. 138 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 4: Select the Lotus Connection for the Data Source Select the Connection for external data source you want to monitor from those available in the Select Connection dialog box, shown below. Note When the list of tables is large, you can scroll down the list to find the table you want to select. However, if the list of tables is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Chapter 12: Virtual Fields Activity 139 When you have selected a connection and there is more than one table in the external database, another database opens asking you to select a table in the external database. See the illustration below. 140 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 5: Map Key and Data Field(s) After specifying the external data source and providing any required connectivity information, such as user name and password, the Data Field Mapping dialog box appears. You must map key fields and data field(s) between the Domino application and the external data source. The illustration below shows the interface for mapping key fields and data fields in a Virtual Fields activity. Note If there are unmapped columns which do not accept NULLs, you will get a NULL assignment error when creating a new record. See Mapping and Unmapped Columns in “Usage Requirements, Performance, and Mapping in Virtual Fields Activities” in this chapter. Note When you create an activity using an OLE DB connection, the key field should not be a TEXT, NTEXT or IMAGE datatype. These datatypes cannot be used in an ORDER BY clause or in the WHERE, HAVING, or ON clause, except with the LIKE or IS NULL predicates. Using the UNID as Key Field LEI provides an option to use the Notes universal ID that is created automatically by Notes for each document when a document is created. This makes it unnecessary for you to include an extra field in your Notes front end solely as a key field. Chapter 12: Virtual Fields Activity 141 Step 6: Select Events to Monitor After mapping key and data field(s), the Virtual Fields Activity Event Selection dialog box appears. The last required step is to select the event(s) (create, open, update, delete) that you want to monitor. You see this interface only when you are creating or editing a Virtual Fields activity document. Note You must select one or more events to monitor. You can select any combination of events. You must select both Open and Update if you want to monitor Update events. You will receive an error message if you just choose Update. Note The events that you are monitoring are on the Notes Form you specified in this activity document, however, when you update, you update to the external system. The user name is used to connect to the external system must have read and write permission for that external system. If a user has only read rights, that user can open a document, but cannot update that document and save changes to the external system. 142 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 7: Name the Activity Document When prompted, enter a unique activity name and click OK. After you name the activity and click OK, the following informational message box appears outlining your options. Click OK and the Virtual Fields Activity document appears, showing your selections. The numbers in the document correspond to the major steps the Assistant uses to create the Virtual Fields activity. Chapter 12: Virtual Fields Activity 143 Step 8: Enable Subforms You can select this option to allow the use of subforms that are controlled by a formula. Subforms that are not controlled by a formula are included by default. If you select this option, a dialog box opens that lets you select from a list of computed subforms. Include only subforms controlled by a formula. 144 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 9: Select Options At this point, you can select the specific options that you want for this Virtual Fields activity. See “Virtual Fields — Options,” in this chapter for a full description. Chapter 12: Virtual Fields Activity 145 Step 10: Select Scheduling Options Choose how to control scheduling of the Virtual Fields activity. The options are Manual, AutoStart, and Custom. See the “Scheduling” section in the chapter entitled “Introduction to LEI Activities” for more information. Step 11: Save, Initialize Keys, and Close the Document You must initialize keys once and only once. Choose File - Save to save this Virtual Fields activity definition. This also causes the Initialize Keys button to appear. Click the Initialize Keys button. You can now close the Virtual Fields Activity document. Choose File - Close to close the Virtual Fields Activity document and return to the LEI Administrator. Note For more information about Initializing keys, see “Initialize Keys in Virtual Fields” in this chapter. Note If you have already created key documents, do not initialize keys again. Tip If you create a Virtual Fields Activity such that output parameters are being used and the field(s) to receive the data contain Hide When formulas such as the following, you must manually refresh the form to see the Hide When field value: For field FIELD_FOO: Hide When FIELD_FOO="" After you save your document, the Hide When field value will remain hidden until you refresh the form. However, if you add the following code to your form design Postsave event, using Domino Designer, the Hide When field value will be visible immediately after you save your document, with no manual refresh required. Sub Postsave(Source As Notesuidocument) Dim workspace As New NotesUIWorkspace Dim uidoc As NotesUIDocument Set uidoc = workspace.CurrentDocument Call uidoc.Refresh End Sub 146 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 12: Process the Virtual Fields Activity You can process the activity using the Start and Stop buttons in the Action bar of the Activities view or the Favorites (Folders) view. Note When you select “Auto Start” in the Scheduling section of the activity document, the Virtual Fields activity is established each time that the server starts. See the Scheduling section for more information. Click the Virtual Fields Activity to select it and then click the Start button to begin processing it. Note To stop a Virtual Fields Activity, select the Virtual Fields activity and click the Stop button. Creating a Virtual Fields Activity without the User Assistant You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Fields Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. If you disable the User Assistant, a blank activity document is displayed, which you can fill in to define the activity. This section describes how to create a Virtual Fields Activity without using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” Step 1: Open the Activity Document Click the Add Activity icon. The Virtual Fields Activity Document appears. Step 2: Enter a Name for the Activity Enter a unique name for the activity in the Name field at the top of the document. Step 3: Select the Domino Application and the Form to Monitor Click the down-arrow icon in the “Domino Application” section. A list of databases appears. Select the database that contains the Domino application that you want to monitor. You are also prompted for a form name. Note If the list is large, you can scroll down the list to find the Domino application (nsf) you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Chapter 12: Virtual Fields Activity 147 Note Immediate or “real-time” monitoring uses the Notes replication ID of the application database. A Virtual Fields activity does not distinguish between the databases that share the same replication ID. Databases have the same replication ID when they are replicas of each other and when one database is created using the operating system to copy an existing database. Step 4: Select the Lotus Connection for the Data Source Click the down-arrow icon in the External System section to select from existing Lotus Connection documents. Note When the list of tables is large, you can scroll down the list to find the table you want to select. However, if the list of tables is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Step 5: Enable Subforms Select this option to allow subforms controlled by a formula to be used. Subforms that are not controlled by a formula are included by default. If you select this option, a dialog box opens that lets you select from a list of computed subforms. Include only subforms controlled by a formula. 148 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 6: Mapping Key and Data Fields After specifying the external data source and providing any required connectivity information, such as User Name and Password, the Data Field Mapping dialog box appears. You are required to map key fields and data field(s) between the Domino application and the external data source. Click the down-arrow icon in the “Mapping” section. The field mapping dialog box appears. Use this dialog box to map the key and data fields between the Domino application and the external data source. The illustration below shows the interface for mapping key fields and data fields in a Virtual Fields activity. Note If there are unmapped columns which do not accept NULLs, you will get a NULL assignment error when creating a new record. See Mapping and Unmapped Columns in “Usage Requirements, Performance, and Mapping in Virtual Fields Activities” in this chapter. Note When you create an activity using an OLE DB connection, the key field should not be a TEXT, NTEXT or IMAGE datatype. These datatypes cannot be used in an ORDER BY clause or in the WHERE, HAVING, or ON clause, except with the LIKE or IS NULL predicates. Chapter 12: Virtual Fields Activity 149 Using the UNID as Key Field LEI provides an option to use the Notes Universal ID that is created automatically by Notes for each document when a document is created. This makes it unnecessary for you to include an extra field in your Notes front end solely as a key field. Step 7: Select Event(s) to Monitor If you are filling out a Virtual Fields form, you must select one or more events to monitor. Click the down-arrow icon in the Events section. A dialog box appears. Select the document events that you want to monitor. You can select any combination of events. Note You must select both Open and Update if you want to monitor Update events. You will receive an error message if you just choose Update. Step 8: Select Options and Event Options Select the options and the event options you want in the “Options” and “Event Options” tabs in the document. See the section “Virtual Fields Activity Options and Event Options” later in this chapter for more information. Step 9: Select Scheduling Options Select how you want to control the scheduling of the Virtual Fields activity. The options are Manual, AutoStart, and Custom. See the “Scheduling” section in the chapter entitled “Introduction to LEI Activities” for more information. Step 10: Save, Initialize Keys, and Close the Document You must initialize keys once and only once. Choose File - Save to save this Virtual Fields activity definition. This also causes the Initialize Keys button to appear. Click the Initialize Keys button. You can now close the Virtual Fields Activity document. Choose File - Close to close the Virtual Fields Activity document and return to the LEI Administrator. Note If you have already created key documents, do not initialize keys again. For more information about Initializing keys, see “Initialize Keys in Virtual Fields” in this chapter. 150 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Tip If you create a Virtual Fields Activity such that output parameters are being used and the field(s) to receive the data contain Hide When formulas such as the following, you must manually refresh the form to see the Hide When field value: For field FIELD_FOO: Hide When FIELD_FOO="" After you save your document, the Hide When field value will remain hidden until you refresh the form. However, if you add the following code to your form design Postsave event, using Domino Designer, the Hide When field value will be visible immediately after you save your document, with no manual refresh required. Sub Postsave(Source As Notesuidocument) Dim workspace As New NotesUIWorkspace Dim uidoc As NotesUIDocument Set uidoc = workspace.CurrentDocument Call uidoc.Refresh End Sub Step 11: Process the Virtual Fields Activity You can process the activity using the Start and Stop buttons in the Action bar of the Activities View or the Favorites (Folders) View. (When you select “Auto Start” in the Scheduling section of the activity document, the Virtual Fields activity is established each time that the server starts. See the Scheduling section for more information.) Click the Virtual Fields activity to select it and then click the Start button to begin processing it. Note To stop a Virtual Fields activity, select the Virtual Fields activity and click the Stop button. Chapter 12: Virtual Fields Activity 151 Virtual Fields Activity Document Options Once you have specified the Domino application, the connection, and data mapping, there are several options to consider for setting up a Virtual Fields activity. The options are broken down into two major areas: Options and Event Options The Options tab is broken down into several areas or functions. • General Options • Multi-value Data • Integrated Credentials • Virtual Attachments The Event Options tab is broken down into an optional set of areas. • Create • Open • Update • Delete See the chapter entitled “Introduction to LEI Activities” for a description of the following sections that are common to all Activity documents: • Activity Name • Scheduling • Category • Comments Note A Virtual Fields activity requires that you initialize keys or that “key documents” be created in some way. Initializing keys creates key documents based on the key field(s) in the LEI activity document. These key documents serve as data containers after the activity is enabled. Initialize keys only once. See “Initializing Keys in Virtual Fields Activities” in this chapter for more information. Note If after upgrading DECS, or upgrading from DECS to LEI, you discover you are unable to get data when you are running Virtual Fields activities, you may have to open each Virtual Fields activity document and ensure that the Domino Server chosen in the activity document is correct. 152 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options in Virtual Fields The General Options section of the Virtual Fields activity document, illustrated below, presents options that can be applied to an activity regardless of the type of Event the Virtual Fields activity monitors. Each of the General Options is described after the illustration. General Options Description Monitor Order If you use more than one Virtual Fields activity for a single Notes form, you can specify the order in which the document’s events are intercepted. The monitor order also enables you to use multiple activities that connect to different tables and use values found by the first Virtual Fields activity (monitor order 1) as keys for subsequent Activities (monitor order 2, 3, etc.). Note Activities with monitor orders of 2, 3, 4, and so on, must monitor an Open event. They may monitor other events too, but one of the events must be an Open event. Depending on the nature of the activity, the activity with monitor order 1 may also be required to monitor an Open event. continued Chapter 12: Virtual Fields Activity 153 General Options Description Max. Connections This option sets the maximum number of connections to the external database that can be open to service concurrent user requests simultaneously. LEI opens one connection to the external data source when the first Domino Application event occurs. If two or more events occur simultaneously, additional connections are made, up to the maximum number of connections specified by this option. When the maximum number of connections is reached, subsequent events are queued and occur when each preceding event is serviced. Each connection lasts only as long as necessary to read from or write data to the external data source. While the connection is persistent, the time required to service each event is minimal and depends on the amount of data being read or written. The maximum number of connections, therefore, does not need to be that great in order to service multiple events. Lotus recommends that you set the maximum connections to 2 or 3, and if users experience significant delays, you can increase this number. Form Override The default is to monitor only documents using the form specified in the Domino Application section. Selecting this option causes the Virtual Fields activity to process the selected events for all the documents in the Notes database that have the same key(s) as the original metadata, regardless of the form. continued 154 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options Description Filter Formula An optional Notes formula defining the documents that the Virtual Fields activity will monitor. Use this option to cause the Virtual Fields activity to process only documents that satisfy the specified formula. Note When you set up a Filter Formula, the selected field must also be defined in “Saved Fields” under the Data Storage option (see below). Note If you use Virtual Documents as a source for “key documents” in a Virtual Fields activity and you do not map the field(s) that you entered in the Filter Formula option in the Virtual Fields Activity, you must select the “Leave selected real-time fields in documents” option in the Data Storage field of the Virtual Fields Activity. If you do not do this and a record that meets the filter formula is updated, it will be removed from the Notes view because the filter field on the Virtual Document side is updated to NULL. Therefore, these fields must remain in the Virtual Fields Activity. However, if the filter field is mapped as a key field in the Virtual Fields activity, you do not have to keep the selected fields in the Virtual Fields Activity. Keep in mind that any additional fields mapped on the Virtual Documents side will be displayed by the Virtual Fields activity regardless of whether the filter formula was met. If a document is created that does not match the filter criteria, it is not inserted into the external table (hence, it is not “processed”), instead, it is saved locally in the .nsf regardless of the “Data Storage” setting. Data Integrity w Prevent data loss — Enable this option to have LEI write an error to the log for any data that is lost as a result of the transfer and terminate the transfer. w Allow precision loss — Enable this option to have LEI not report loss of numerical or datetime precision as a result of the transfer. This allows some loss of precision without stopping the transfer. This option is the default. w Allow precision loss & truncation of text — Enable this option to have LEI allow precision loss and to truncate text data when necessary to conform to field lengths in the external database. Note that key fields are not truncated. continued Chapter 12: Virtual Fields Activity 155 General Options Description Trim Trailing Spaces This option affects any trailing spaces that exist in the Text fields of the external data. Text trimming only occurs when the fields are read from the external source. w Trim spaces on all non key fields — Select this setting to trim trailing spaces only from data fields, not from the key fields. (Default) w Do not trim spaces on any fields — Select this option to leave trailing spaces in all fields. Trailing spaces may be required to ensure matching of fields between Notes and the external data. Note When data is put into external system fields of fixed length and then retrieved, the connection with the external system may be lost because the external system has padded the data with spaces to fit the fixed length of the field. If this occurs, use either a variable length field (VARCHAR) in the external system database, or enable the “Trim spaces on all non key fields” option. You may encounter this situation when using data of type CHAR as a key field for Oracle 7 or 8 external data sources. When you use a CHAR data type as a key field, use the default setting, “Trim spaces on non-key fields.” Note When using a Virtual Fields Activity that monitors the Create Event, key fields of type character (fixed length) are only effected by the activity’s trim options when keys are first initialized. While the activity is running, newly created character (fixed) key fields will remain exactly as they are entered when creating the new record in Notes. New Line Delimiter for Connection Select the option you want to use to convert the Notes new-line delimiter to a new-line delimiter in another system. continued 156 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options Description Data Storage w Remove all real-time fields from documents — Enable this option if you want to remove all the data fields mapped in the Activity from the Notes document before it is saved to disk. This is the default. w Leave all real-time fields in documents — Enable this option if you want to leave all the data fields in the Notes document, rather than removing them after updating the external source (see above). This option only takes effect when creating or updating a document when the activity is active. w Leave selected real-time fields in documents — Enable this option if you want to leave selected data fields in the Notes document. You may want to select this option in order to enable views of the forms. Saved Fields Click the down arrow to open a list of Notes data fields from which you can select the data field you want. If you do not select any fields, it is equivalent to enabling the "Remove all real-time fields from documents" option. Note If you use the Filter Formula option, the selected field must also be defined here in Saved Fields. Initialize Keys in Virtual Fields When you are working in a Virtual Fields activity document, there is a button labeled “Initialize Keys” in the Action bar. When you click this button LEI populates the currently selected Virtual Fields activity key fields with data from the external data source for that connection. This command should be used only once after creating a Virtual Fields activity. Initializing keys again creates duplicates of existing documents. If you have specified “Leave selected real-time fields in documents” or “Leave all real-time fields in documents,” the associated fields are transferred to the Domino Application. Note When using Initialize Keys with a Virtual Fields activity that has a Filter Formula, you are asked if you want to use a conditional clause to restrict the number of external records imported into Notes. If you choose Yes, you are prompted to enter the external conditional clause. The conditional clause must be in the external system’s syntax for a keyed selection clause. For SQL based systems, the conditional clause is that part of a WHERE clause following the word “WHERE.” For example, if the complete statement were: SELECT * from Table WHERE Number > 1, you would only enter Number > 1 as the key initialization condition. Chapter 12: Virtual Fields Activity 157 Note In a Create event, character (fixed) key fields are unaffected by trim options they are also unaffected by how they are stored in the external data source. These fields remain exactly as they were entered when creating the record in Notes. Caution When you create an activity using an OLE DB connection, the key field should not be a TEXT, NTEXT or IMAGE datatype. These datatypes cannot be used in an ORDER BY clause or in the WHERE, HAVING, or ON clause, except with the LIKE or IS NULL predicates. Multi-value Data in Virtual Fields To enable a LEI activity to work with multi-value data, check the Multi-value Data check box under the Options section of the activity document. This expands the section displaying additional selections. The Multi-value Data options allow you to consider non-key fields as multi-value fields. Once selected, there are more options for sorting and ordering for you to choose from. Each Multi-value Data option is described after the illustration. 158 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Multi-value Data Options Description Multi-value Data Use multi-value data fields — Enable this option to have the Virtual Fields activity consider non-key fields to be multi-values. Multi-value Data allows a single document in a Notes database to relate to multiple records in the connection. When enabled, the Open event will read all records of the external system which match the key data field(s) and combine the multiple fields data into Notes multidata fields. The Create, Update, and Delete events reverse the operation. Note Multi-value data cannot be further controlled with conditional keys such as “Not Equal To” and “Less Than or Equal To”. Subkey Fields This is an additional key field (for one or more fields) that defines the unique key within a collapsed record. If you choose more than one field, you can order them in any way you would like. For example, if four orders by single customers are collapsed into one document with the customer number as the key, and one order is changed, then the order number is required to ensure that the correct record is updated. In this example, the order number is the subkey and provides a unique identification across the individual external records for this customer when it is expanded back to the external data source. Sorting Sort multi-value data — Enable this option to have the Virtual Fields activity sort data within the non-key fields. When enabled, the values within the multi-value data fields will be sorted according to the sequence of the indicated “Sorting fields” and their relationships based on the type of sorting (text, binary and ascending, or descending). Also, when you enable this option, the Text Order, Direction, and Character Set fields become available. Sorting fields Sorting key fields that define the order of the data within a collapsed record. The individual values within the multi-value data field are sorted according to the “Text order” and the “Direction” options. All remaining fields will have their multi-value data appropriately sequenced to preserve their order with those of the sorting field or fields. continued Chapter 12: Virtual Fields Activity 159 Multi-value Data Options Description Text order w Binary — Sort based on the binary order of the character codes. Recommended. w Case Sensitivity — Lower case is sorted before upper case. w Case Insensitivity — Case is ignored in the sorting process. Direction w Ascending — Sorting is done alphabetically from A to Z. Earlier dates are placed before more recent ones; smaller numbers are placed before larger ones. w Descending — Sorting is done alphabetically from Z to A. Recent dates are placed before earlier ones; larger numbers are placed before smaller ones. Character set Force data into the selected character set for the purpose of sorting. By default the underlying Connection’s character set is used. To provide a character set, use the corresponding suffix. For example, for LCSTREAMFMT_IBMCP932, use “CP932.” See an appendix in the Lotus Connector LotusScript Extensions Guide for a list of character sets. Multi-value functionality allows activities to group multiple records with a common key value into a single Notes document. By using multi-value fields, these records can appear, for example, as table data. All records with the same grouping key field value(s) are combined (collapsed) until a different key is encountered. For example, to capture this information, you could use Designer to create a two-row table on a Notes form and adding multi-value enabled fields for multi-values, with separator options set to “new line.” In all LEI activities, a key must be defined as one or more fields, for example, “JOB” might be defined as the key field. In this case, the LEI activity document would have a “Key” value field that is not a unique field. Subkeys An important option that allows changes to be written back to a data source (external system) is the “Subkey fields” option. This option makes it possible for you to select a unique key to identify this individual record in the external system data source. For example, if six employees are combined (“collapsed”) into one document with the job title as the key, and data regarding one employee is changed, then the employee number is required to ensure that the correct record is updated. In this example, the employee number is the sub key and provides a unique identification across the individual external records for this employee when it is expanded back to the external data source. 160 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Also note that there are numerous sorting options as well. These options allow sorting on the data contained in the combined (“collapsed”) records. Note You should use extreme caution when you update an existing subkey. You should not, under any circumstances, update it to an existing subkey. If you have a unique constraint on the subkey column and you modify a subkey to a value that does not yet exist in the current record’s primary key, you will get an insert error. If you do not have a unique constraint, you will not get an error and the update will proceed normally. If, however, you update the subkey to another that already exists for the same primary key, you may loose data even if you have a unique constraint on that column. Integrated Credentials in Virtual Fields User credentials for the external system are automatically looked up when the Integrated Credentials options is selected. The credentials are used to connect to the external system. This increases Domino security by integrating it with the security already in place for the external system. Note The integrated credentials database is described in the Introduction to Activities chapter of this document. See the section entitled “Integrated Credentials Database” for information about setting up and using integrated credentials. Each of the Integrated Credentials Options is described in the table following the illustration. Chapter 12: Virtual Fields Activity 161 Integrated Credentials Options Description Use Connector Credentials Uses the user ID and password that is specified in the Connection Document you are using in this activity. Obtain Credentials from Data When you select this option the form expands to include the following options: w Missing Credentials — You can select the Use Connector Credentials option. If you select this option, then in case of a credentials lookup failure, default credentials in the activity can be used. If you do not select this option, the operation can fail (this is the default behavior). w Credentials UserId field — The name of the field that contains the user ID to use to connect to the external system. w Credentials Password field — The name of the field that contains the password to use to connect to the external system. w Connection Cache Size — Maximum number of distinct credentials active at one time. Note If you use Virtual Documents as a source for “key documents” in a Virtual Fields activity, you must put the credentials ID and password in the Virtual Documents Activity Document, putting them in the Virtual Fields Activity Document has no effect in this case. Lookup Credentials When you select this option the form expands to include the following options: w Missing Credentials — In case of a credentials lookup failure, either the operation can fail (this is the default action) or default credentials in the activity can be used. w Credentials DB Filepath — The name of the credentials database to be used for storing Notes user names and corresponding external systems credentials. The path is specified relative to the LEI directory (for example leidir). Note You must have created this database using the template leicred.ntf. See “Setting Up and Using Integrated Credentials” in the Introduction to Activities chapter of this document. w Connection Cache Size — Maximum number of distinct credentials active at one time. 162 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Integrated Credentials and Full Text Indexing In order to perform full text indexing with Virtual Documents or Virtual Fields in conjunction with integrated credentials, you must add the server ID to the integrated credentials database. Integrated Credentials and Server to Server Replication In order to perform server to server replication in conjunction with integrated credentials, you must add the server ID of the server being replicated with to the integrated credentials database. For example, if you want to replicate a Virtual Documents or Virtual Fields-enabled database on Server A to Server B, you must add Server B’s server ID to the integrated credentials database on Server A. Integrated Credentials in a Domino Cluster The server IDs for all the servers in a Domino cluster that contain replicas of the Virtual Documents or Virtual Fields-enabled database must also be added to the integrated credential database. For example, if a Virtual Documents-enabled database on Server A has replicas on Servers B and C, and Servers A, B, and C are part of the same Domino cluster, the server IDs for Servers B and C must be added to the integrated credentials database on Server A. If this is not done, the cluster replication process will fail to keep the replicas synchronized. Virtual Attachments The Virtual Attachments feature allows you to use Notes to add attachments to the external system database. File attachments will be stored in the external system rather than in your Domino application. Once added, they can be accessed through the virtual views. Note Storing the file name and Notes ID makes identification and retrieval of the attachments easier but requires additional columns in the external system table. The list of required and optional Virtual Attachment columns is as follows: Column Name Required or Optional EIDBID Required EIFILEID Required EIFILESIZE Required EIUNID Optional EIFILENAME Optional EICONTENTS Required Chapter 12: Virtual Fields Activity 163 You do not need to include the EIUNID or EIFILENAME fields in the attachments table. They are not needed by LEI, but LEI maintains them if they are provided. These fields allow external database applications to more easily recognize and access the data outside of LEI/Domino. EIUNID is the noteid value that corresponds to the EIUNID field in the row of the data/key table. EIFILENAME is the name of the attachment as specified in the note. You can control this functionality using the Store Optional Fields option on the Activity Document. An important restriction is that the virtual attachments table must be in the same external system and in the same database as the external system data being accessed by the activity. The virtual attachments table cannot reside in a different external system or database than the one being used by the Virtual Documents Activity. This is because the same Connection Document is used to access the external system. The Virtual Attachments feature allows you to add attachments to virtual documents as you would to a native Notes document. The exception is that file attachments are stored in the external system rather than in your Domino application. The attachment data is stored in a binary field in an external system table that you create. Other columns in the external system table store key data, which enables the activity to locate a specific attachment. A single table can be used for multiple Virtual Documents Activities. The following considerations are helpful as you use this feature. • You must select the Virtualize Attachments option when you use attachments in a Virtual Document. You cannot save an attachment in a Virtual Documents Activity unless it is virtualized. If you do not, your attachments will not be saved. • If the “fixup” utility is run on the Notes database while a monitored document is open, you must either close and reopen document or save the document to access any restored Virtual Attachments. For information about “fixup” see the Domino server documentation. 164 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Attachments Options Description Attachments Virtualize Attachments — You must select this option to be able to store attachments with your virtual documents. Enabling this option requires that you create and specify an external system table to contain the contents of the attachment. Note You must specify an attachment table in order to create a virtual attachments table. Attachment table Specify a separate table for the contents of the attachments. In addition to the contents of the attachments, this location also contains a locator key. Optionally, the file name and Notes ID may be stored as an additional data element, select “Additional data” for this option. Note When you create a virtual attachment table in Sybase, the column names must be in upper case. Note See the section entitled “Creating a Virtual Attachments Table” for complete step-by-step instructions. Additional data Select this option to add the attachment filename and its Notes ID to the attachment table. This option is not required. Note Storing the file name and Notes ID makes identification and retrieval of the attachments easier but requires additional columns in the external system table. Indexing Fields for Increased Performance If you are using Virtual Attachments, the Virtual Attachment Table should be indexed on EIFILEID. In addition, the Virtual Attachment Table EIDBID column should be indexed if the table supports multiple databases. It is used by multiple Virtual Documents activities, some of which monitor forms from different Domino databases. Fields and Data Types The following table describes the necessary fields and data types that are required for the attachment table at the external system as well as the fields that are required on the Notes form. Chapter 12: Virtual Fields Activity 165 Attachment Table External System Data Types Fields Oracle Informix Sybase [a] DB2 [b] SQL Server Access 2000 EIDBID** (required) VARCHAR2 CHAR (16) (16) or CHAR (16) CHAR (16) CHARACTER CHAR (16) (32) TEXT (16) EIFILEID** (required) NUMBER INT INT INTEGER INT NUMBER INTEGER EIFILESIZE [c] (required) NUMBER (8) INT INT INTEGER INT NUMBER LONG NTEGER EIUNID [d] (optional) CHAR CHAR (32) (32) or VARCHAR2 (32) CHAR (32) CHAR (32) CHAR (32) TEXT (32) EIFILENAME (optional) VARCHAR2 VARCHAR (255) (255) VARCHAR 255) VARCHAR (255) VARCHAR (255) TEXT (255) EICONTENTS required) LONG RAW BYTE IMAGE BLOB (1048576) IMAGE OLE Object ** Index these fields at the external system to increase performance. a) In case sensitive RDBMS’s, such as Sybase, column names must be in upper case, for example, EIUNID. b) DB2 — with up to a 1MB attachment. c) EIFILESIZE column length may be adjusted, but it must remain an integer (X,0). d) EIUNID is a DWORD, or an unsigned 32-bit integer. According to the formula 2^n - 1, this gives a maximum EIUNID of 4,294,967,295 which requires 10 digits. Specifying the Attachment Table and Schemas By default, if a schema is not specified for the virtual attachment table, the user making the connection is used by default. For example, if the user is TESTONE and the schema is DBSOURCE, the user will be taken as the schema, TESTONE.MY_TABLE. If the virtual attachment table is picked from the selection dialog, the schema name is added as a prefix. However, if the user types the table name in by hand and there is a difference between the schema and the connection user, he must type the schema name as a prefix, for example, DBSOURCE.MY_TABLE. 166 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Any attachments that were not entered into the external system through the Virtual Attachment process will not be accessible through either Virtual Fields or Virtual Documents. For example, if you added attachments to the external system database through another process, for example SQL commands, you cannot access them through Virtual Fields or Virtual Documents. You can create the table using SQL commands. Creating a Virtual Attachments Table In the Virtual Documents and Virtual Fields Activity Documents, you can configure the activity to use a relational table to store file attachments as binary objects by creating a virtual attachments table. 1. Click the “Virtual Attachments” tab in the middle of the Activity Document. 2. Click the Attachments: Virtualize Attachments option. Chapter 12: Virtual Fields Activity 167 3. In the Attachment table field, enter the name of the table that you want to create. 4. Optionally specify that you wish to store optional fields. 5. Click the Create Virtual Attachments Table button. 6. The Create Attachment Table dialog appears, enabling you to select the options that will set an optimal size limit for some fields and create indexes as appropriate for others. Complete the form using the descriptions presented below. • Table Name — This field displays the table name that you entered in the “Attachment table name” field of the Activity Document. Note You must include the owner name in the first part of the Table Name entry. For example, to create a table named EMPTRANSFER with an owner name of DB2ADMIN, enter DB2ADMIN.EMPTRANSFER in the Table Name field on the Create Attachment Table form. 168 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • Use Connector Credentials — You need a relational database user ID with privilege to create metadata. If the user ID in the Connection Document does not have this privilege, uncheck this box and enter a different user ID and password. 7. When you have completed the form using the dialog box descriptions above, click OK in the upper right of the form to continue. 8. A message appears, indicating success. Note If the table already exists, you will prompted whether to overwrite it. If you answer Yes, the existing table and all the data in it are erased. Since most connection types don’t allow existing tables to have fields added or field properties adjusted by an external program, changing existing attachment tables that were incorrectly created is not supported. For information about which fields will be indexed by the table creation tool, see the section in this chapter entitled “Virtual Attachments in Virtual Documents Activities.” Event Options Create, Open, Update, and Delete The monitored events are events that occur in the monitored Notes form that you specified in this activity document. The Create, Open, Update, and Delete sections of the Virtual Fields activity document, provide many options that depend on the type of Event the Virtual Fields activity monitors. Each of the options is described in the tables on the following pages. Chapter 12: Virtual Fields Activity 169 You must choose which events you want the activity to monitor. The options are displayed specific to the type of event or events you choose to monitor. You must choose at least one event to monitor. The document is illustrated below (in this illustration all event options have been selected). Create Options When monitoring Document Create Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. 170 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Create Options Description Pre-Create Formula This option executes a Notes formula language statement before the document data is stored. You can use this option to modify existing fields or to compute values for additional fields. For example: FIELD LASTNAME:=@if(LASTNAME = ““;“NA“;LASTNAME);”” Note If you are connecting to an Oracle external system and you must use a text field(s) of fixed length as a key field(s), use a formula filter to pad the specific field to the correct length. See the Lotus Connector for Oracle 7 or 8 information in Lotus Connectors and Connectivity Guide for more information. Stored Procedure This option executes a stored procedure in the external data source to store data that has been entered into the Notes document. The field(s) are supplied to the stored procedure as input parameters. Domino/LEI supports stored procedure browsing. To specify the stored procedure and see the parameters that will be passed to the stored procedure and their order, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. In a Virtual Fields activity this includes the specified key field(s). Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. Stored Procedure Output Enable Procedure Output — This option is available only if a stored procedure has been specified in the “Stored Procedure” field. When this option is selected, it opens up the “Stored Procedure Output Mapping” section of the activity document, where you enter input and output field names. This option enables the return of data from a stored procedure in the external system to the Domino Application (that is, the Notes document). The activity’s key(s) and field(s) are supplied to the stored procedure as input parameters. To see the arguments that will be passed to the stored procedure, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Note Currently, only the Lotus Connector for DB2 and the Lotus Connector for SAP (available for purchase separately) support output parameters. continued Chapter 12: Virtual Fields Activity 171 Create Options Description Stored Procedure Output Mapping w Domino Application — Specify the fields in the Domino Application (Notes document) that will receive the stored procedure output. w Lotus Connection — Specify the fields from the external system that contain the stored procedure output. The first field from the external system maps to the first destination field, the second external system field to the second destination field, and so on. Note These fields must not be specified in the Mapping section of the activity document. Open Options When monitoring Document Open Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Open Options Description Pre-open Formula (Virtual Fields only) This option executes a Notes formula before the document data is retrieved from the external system. This option provides you with an opportunity to compute additional fields or modify existing fields used as key information, thereby altering data selection. Post-Open Formula This option executes a Notes formula immediately after data is retrieved from the external system. You can use this option to compute additional fields or modify existing fields. For example: FIELD FIRSTNAME := @if(FIRSTNAME = “Al“;“Albert“;FIRSTNAME);”” Note You should not use Post-Open Formulas when using conflict detection. continued 172 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Open Options Description Stored Procedure This option executes a stored procedure in the external data source to determine the data that will be retrieved into the Notes document. To see the arguments that will be passed to the stored procedure, type the name of the stored procedure in this field or select the stored procedure from the drop down list and then press F9. Domino/LEI supports stored procedure browsing. In Virtual Fields, the specified key fields are displayed next to the stored procedure name. For Lotus Connectors that support procedure output parameters, the key fields (in Virtual Fields activities) are passed as input/output parameters and the other fields are passed as output parameters. Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. Missing External Records (Virtual Fields only) Stored Procedure Output Mapping If no matching external record is found for a document on open, you can choose what the action should be: w Create Record — This option creates a new record in the external database. w Generate Error — This option generates a Notes error message box. w Ignores — This option ignores the fact that there are no matching record and opens the document even though it contains no data. These fields are visible only if “Stored Procedure Output” in either a Create or an Update event has been selected, see either Create or Update event for more information. Chapter 12: Virtual Fields Activity 173 Update Options When monitoring Document Update Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Update Options Description Pre-Update Formula When a Notes document is updated, this option executes a Notes formula statement before the data is saved in the external database. Stored Procedure When a document is updated, this option executes a stored procedure in the external data source to update the data related to the document. The field(s) are supplied to the stored procedure as input parameters. To specify the stored procedure and see the parameters that will be passed to the stored procedure and their order, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Domino/LEI supports stored procedure browsing. In a Virtual Fields activity this includes the specified key field(s). Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. continued 174 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Update Options Description Stored Procedure Output Enable Procedure Output — This option is available only if a stored procedure has been specified in the “Stored Procedure” field. When this option is selected, it opens up the “Stored Procedure Output Mapping” section of the activity document, where you enter input and output field names. This option enables the return of data from a stored procedure in the external system to the Domino application (the Notes document). The activity’s key(s) and field(s) are supplied to the stored procedure as input parameters. To see the arguments that will be passed to the stored procedure, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Note Currently, only the Lotus Connector for DB2 and the Lotus Connector for SAP (available for purchase separately) support output parameters. Conflict Detection Check External Data for Changes — This option ensures that the external data has not changed since the Notes document was opened. If it has changed, an update to the external data source will fail. If this option is enabled and if you make changes to data in a Notes document and then save the document, you must exit the document before making any more changes — even though you have saved the document. Note “Conflict Detection” is not supported when you use an HTTP server to access documents. Note In a Virtual Fields activity, if you turn on “Conflict Detection” and you connect to a Sybase external system, the Sybase external system table must be indexed. Field Level Updates Only Update Changed Fields — This option causes the Virtual Fields activity to not update fields in the external data source unless the corresponding fields in the Notes document have been edited. Use this option if you have triggers in the external system that monitor updates to individual columns of a table. continued Chapter 12: Virtual Fields Activity 175 Update Options Description Key Field Updates (Virtual Fields only) The following three settings are available for key field updates: Stored Procedure Output Mapping w Block: Do not allow updates to key fields in the Notes document or the external data source records. w Delete/Insert: Updates to key fields will cause the original record in the external data source to be deleted and a new record with the new key added. w Ignore: Do not update the key fields in the external data source. If a key field is edited, the change will be stored with the Notes document but the key field(s) in the external data source will not change. w Domino Application — Specify the fields in the Domino Application (Notes document) that will receive the stored procedure output. w Lotus Connection — Specify the fields from the external system that contain the stored procedure output. The first field from the external system maps to the first destination field, the second external system field to the second destination field, and so on. Note These fields must not be specified in the “Mapping” section of the activity document. Delete Options When monitoring Document Deletion Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. 176 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Deletion Options Description Pre-Delete Formula Notes formula language statement to execute on the Notes document immediately prior to the deletion in the external data source. Here is an example that sends an E-mail containing information about a deleted document: FIELD LASTNAME:=LASTNAME; FIELD FIRSTNAME:=FIRSTNAME; @MailSend(“John Q Public“;““;““;“Doc Deleted“;“Doc deleted from MyDatabase w/ Firstname/Lastname of:“;FIRSTNAME:LASTNAME);”” Note It may be necessary to specify that activity data be saved in the Notes document; otherwise, the specified formula may access the field values after they’ve been deleted. If this were the case for the example above, FIRSTNAME and LASTNAME would be empty. To specify that activity data be saved, in the General Options section of the activity form go to “Data Storage,” choose one of the options “Leave all/selected realtime fields in document.” Stored Procedure When a document is deleted this option executes a stored procedure in the external data source to remove the data related to the document. To see the arguments that will be passed to the stored procedure, type the name of the stored procedure in this field or select the stored procedure from the drop down list and then press F9. Domino/LEI supports stored procedure browsing. In Virtual Fields, the specified key fields are also displayed next to the stored procedure name. In Virtual Fields, only the NOTEID field is displayed. For Lotus Connectors that support procedure output parameters, the key fields (in Virtual Fields activities) are passed as input/output parameters and the other fields are passed as output parameters. Note See the “LEI and Data” appendix for error situations you may encounter using stored procedures that return a result set with Sybase or OLE DB. Stored Procedure Output Mapping These fields are visible only if “Stored Procedure Output” in either a Create or an Update event has been selected, see either Create or Update event for more information. Chapter 12: Virtual Fields Activity 177 Running Virtual Agents Against Virtual Fields Changes made to Virtual Fields documents by running a Virtual Agent will produce updates in the specified documents immediately. However, if the Virtual Agent is updating values contained within a View column, that view will not reflect the changes made to the documents, and pressing Shift F9 will not refresh the view. In order for Virtual Fields to be displayed within a view, you must have selected “Leave all/selected Real Time fields in document” when building the activity. This, in effect, makes native copies of the fields within the document. These fields will not be updated or reflect changes made in the external system until the Notes document is updated manually or by using a formula or script. 178 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 13 Virtual Documents Activity There are three Advanced RealTime Activities in LEI. • Virtual Fields (previously known as LEI RealTime or DECS RealTime) • Virtual Documents • Virtual Agents This chapter provides information about the Virtual Documents Activity. Introduction to Virtual Documents Activities Virtual Documents Activities monitor Notes forms. LEI connects these forms to external data sources (such as DB2) using Lotus Connectors. Once a system administrator has created a Virtual Documents Activity, Domino users can open, create, update or delete external data directly and transparently through their Notes client. By extension, Web clients may open the same Notes form that the activity monitors by accessing a Domino Release 6 server and obtain access to supported external source data. For example, if the external database to be queried or updated from the Notes form is a DB2 database, Notes end-users may work with DB2 data as if it were in Notes. DB2 connectivity software is not required on the client system, however, it must be installed on the Domino server machine. Network access to the external data source is handled by the Domino server machine, which contains LEI connectivity software for the external data source, such as DB2. You will be required to add certain fields to the external system to accomplish this functionality. Comparing Virtual Documents and Virtual Fields A Virtual Documents Activity makes it possible to have documents that are stored in the external system appear to reside in Domino. The functionality provided by a Virtual Documents Activity is similar to that provided by a Virtual Fields Activity. The important distinction between Virtual Fields and Virtual Documents Activities is that Virtual Documents do not require the presence of “key documents” (previously referred to as “stub documents”) that need to have the key fields initialized. Virtual Fields do require these documents and require key field initialization. Another 179 important distinction between Virtual Fields and Virtual Documents is that Virtual Documents data fully participates in View operations. A Virtual Documents Activity stores information that determine the note’s unique identity in the external system itself. Together, this information provides the unique keys to the external system records and are fully managed by LEI. Synchronization issues, which may arise in Virtual Fields Activities between key documents (generated using the Initialize Keys option) and the corresponding external system records, do not exist with Virtual Documents Activity because a Virtual Documents Activity does not generate or use key documents. Like Virtual Fields Activities, the Connection Document for the selected external system must be created before you create the Activity Document. You create a Virtual Documents Activity Document that specifies which Lotus Connector to use to access the external system and which Notes database and form to monitor. Like all other LEI functionality, once a Virtual Documents Activity is defined it is controlled by the LEI Administrator. Also like Virtual Fields Activities, the fields in the form monitored by the Virtual Documents Activity are mapped to external system table columns when you create the activity. The mapping can be edited any time that the activity is not running. The LEI Administrator provides browsing functionality to allow you to easily map an external system table and columns to the corresponding Notes database/form and fields. Unlike Virtual Fields, you do not specify key fields because unique keys will be generated and stored in the external system. For each Virtual Documents Activity, only one form can be monitored. For each form, only one Virtual Documents Activity can monitor it. Note All fields on the form must be mapped. Since none of the documents resides in the .nsf file, all field data is retrieved from the external system database. Any fields that are not mapped to external database fields will not be stored. The use of Virtual Documents in combination with Virtual Fields is described in the “Using Virtual Fields with Virtual Documents” appendix. Advantages of Using Virtual Documents Activities There are several advantages to using Virtual Documents over Virtual Fields Activities. • No component of the actual Notes document is stored natively in the Domino application, however, all the data is available to a Notes user and is indistinguishable from data actually stored in Notes. 180 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • You are not required to maintain key documents in Notes applications, making it more scalable and easier to maintain. • Virtual Documents fully supports views and view refreshes. By contrast, Virtual Field data will not appear in a Lotus Notes views unless the user elects to leave selected non-key Virtual Fields monitored fields in the key document, these could potentially contain stale data even after refreshing the view. • Virtual Document Activities support background virtualization of records added to the external system. This process automatically adds new records that have been added to the external system to views. It also virtualizes all records in the existing data table that have not been virtualized. The following illustration shows the advantage of Virtual Documents over Virtual Fields. In a Virtual Fields Activity there are numerous “key documents” present. These often yield a large Notes database. However in a Virtual Documents Activity, there are no key documents. The Virtual Documents Activity co-manages the Notes database along with Domino and creates “virtual” documents as needed. When the Virtual Document Activity is not running, you cannot access any of these documents or their associated external data. Chapter 13: Virtual Documents Activity 181 Using Virtual Fields and Virtual Documents Concurrently You can concurrently use Virtual Fields Activities with Virtual Documents Activities in the same Notes database. It is even possible to monitor the same Notes Form with a Virtual Documents Activity and one or more Virtual Fields Activities at the same time. In this scenario, the Virtual Documents Activity first builds the Virtual Document from the external system data, and then the Virtual Fields Activity uses a key(s) from the Virtual Document to further populate the document from potentially another external system data source. This makes it possible to use some of the field level flexibility, and monitor order capability, of Virtual Fields Activities but without the need for key documents. See the appendix “Using Virtual Documents with Virtual Fields” for a complete discussion of this topic. Virtual Documents and Public Encryption Keys If any fields in a Virtual Document have been enabled for encryption, and a Public Encryption Key is provided in the document properties security tab, any and all encryption enabled fields will be encrypted as with non-virtual documents. The corresponding data in the external system table will be nulled. The encrypted data will only be visible to Notes users with the appropriate encryption key. External system table columns which are mapped to encryption enabled Notes fields should not have constraints preventing NULL values or uniqueness constraints. Encrypting many fields may necessitate the size of the EINOTEPROPS binary Virtual Document column to be increased beyond the typical recommended value of 10KB. This is particularly true if Rich Text fields or large text fields are encrypted. If Rich Text fields are encrypted, the size of EINOTEPROPS should be increased by the same size as the binary datatype column which is mapped to the rich text field. For example, if an encrypted rich text field “memo” is mapped to a corresponding external system table binary datatype column called “memdata” with a size of 5KB, EINOTEPROPS should be 15KB in size (the recommended 10KB size plus 5KB to hold the encrypted data from the memo field). These represent “rules of thumb”, and depend on the number of encrypted fields, whether there are any encrypted rich text fields, the total number of fields and other factors. See the section entitled, “Required External System Fields for Virtual Documents Activities” for more information about EINOTEPROPS. 182 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Documents and Monitored Forms When creating a Virtual Documents Activity, you must specify which Notes form the activity will be monitoring in your Domino database and specify the mapping between the form fields and the external system table columns. The connectivity information to the external system is defined in a separate Connection Document, which must be created separately before creating a Virtual Documents Activity. With the activity running, you can then create a new Notes document using the monitored form and save it; a new record will be created in the external system table containing the data you entered along with the extra information needed by Notes to treat this record as if it were a “normal” Notes document. When you later open the document it will be displayed using the correct Notes form. Any Notes database links, doc links, attachments, and so on, that you create remain intact in the Virtual Documents. Views containing view selection formulas will correctly display or not display your Virtual Documents, as required by the selection formula. Virtual Documents can be opened, updated, deleted, or created as long as the Virtual Documents Activity is running. Updating or deleting Virtual Documents will perform the corresponding action in the external system table. If you create a document with a different Notes form, or shut down the Virtual Documents Activity, a document is still created but it will be a native Notes document and no corresponding record will be created in the external system table. Similarly, if you attempt to open a Virtual Document being displayed in a Notes view but the Virtual Documents Activity is no longer running, you will not be able to open that document. Refreshing the view while a Virtual Documents Activity is not running causes all of the non-native document information to disappear from the view. Chapter 13: Virtual Documents Activity 183 If another external system client (non-Notes) alters the external system table data, changes will not become immediately apparent in the Domino application until the Notes view is refreshed. Use Shift + F9 to refresh a view if data is altered in this way. The same is true of newly created data. If another external system client creates records, the newly created records are automatically virtualized (if you enable the “Virtualize External System Data” option in the General Options sections of the Activity Document) and will appear in the view when it is refreshed. If records are deleted through another external system (non-Notes) client, they will be removed from the view when refreshed. However, it is recommended that deletions are performed with the Notes client through the Virtual Documents Activity. This helps ensure the integrity of other Domino processes, such as replication, as it concerns Virtual Documents. For more information, see the section entitled “Special Consideration when Deleting Virtual Documents”. Multiple Virtual Documents Activities can run against a single Domino application (.nsf), each monitoring a separate Notes form. Virtual Documents Activities can use separate connections and external system tables, potentially on completely different external systems (for example, DB2 and Oracle). The Notes form information is preserved so the Virtual Document data will always be displayed using the same form from which it was created. Note An external system table can only contain Virtual Documents data for a single .nsf and form. Virtual Documents Activities monitoring different .nsf files and forms must use different tables. This applies to both data tables and key tables. Virtualizing Preexisting Data A special situation occurs when the external system table to be used with a Virtual Documents Activity contains preexisting data or when new rows are added to the table from outside of Notes. A powerful feature of Virtual Documents is the ability to virtualize this data “on the fly”, thus making it available to your Domino application. To enable this ability, select the “Virtualize External System Data” option in the General Options section of the Virtual Documents Activity Document. It is selected by default. When the activity is started, any new rows in the external system will be virtualized before the activity is available (before the activity started message is printed to the log and/or console). If you are starting an activity against external system data with a large number of new rows (for example, virtualizing a large database for the first time), this process could take several minutes or may take more than one hour. 184 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide After the activity is started, any new records added to the external system will be virtualized periodically, based on the Refresh Frequency value specified in the Activity Document. Newly virtualized records are automatically assigned the Notes form that the Virtual Documents Activity is monitoring and will use that form when they are opened. Understanding Virtualization Processing Virtualization occurs as a background task. Initial virtualization occurs when the activity is started. Activity status is reported to the activity log. The log indicates when the virtualization is started, when it completed, and the number of records that were successfully virtualized. Although the log states that the virtualization process has started, it is not actually enabled until the initial virtualization process is complete. Attempting to access or create virtual documents before this initial cycle is complete will yield an error message. Virtualization errors will appear in the activity log if there are problems during the background virtualization. Required External System Metadata for Virtual Documents Activities A Virtual Document Activity does not require key documents in the same way that a Virtual Fields Activity does. However, it does require that there be a table in the external system with four special columns that map to Domino-generated control fields. There are two ways to do this. • Add the four control data columns (described below) to the external system data table directly. This is the table that will be mapped to the Notes form in the Virtual Documents Activity. • Create a separate table, called an External Key Table, at the external system that contains only the four control data columns plus a foreign key field(s) to map to the actual data table. Note You can use the Table Creation Utility button to create this separate table (External Key Table) automatically. See “Using the Create External Key Table Option” in this chapter for complete information. If you elect to use an external key table to store the Notes control fields, the key table must reside in the same database and have the same user access rights as the data table. Chapter 13: Virtual Documents Activity 185 If you select the second method, you specify the name of the External Key Table in the Virtual Documents Activity form and the key(s) used to map the key table to the actual data table. These keys can be made up of a combination of fields. The advantage to this method is that you do not need to alter an existing data table. In either method, the additional four control data columns are automatically mapped to corresponding internal Notes fields when the Virtual Documents Activity is created. You do not need to map them in the Mapping section of the Activity Document. They do not even appear when you enable manual mapping. There is nothing you have to do to ensure their being mapped other than be certain that the names and data types are correct. The four additional Notes control data column names required are as follows: • EINOTEID • EIUNID • EIMODIFIED • EINOTEPROPS Theses column names must not be altered. Sybase users must name these columns using all upper case characters. For details regarding setting up an External Key Table, see the following section “Creating an External Key Table with Virtual Documents Activities.” Note The external system column names, whether added to the existing data table or created in an external key table, must match these names exactly and must be of the appropriate data types. See the section below entitled “Virtual Notes ID Data Type Correspondence for Various Databases” for the appropriate data type for a specific external system. Note The information that LEI writes to the EINOTEPROPS field is derived from internal Domino fields. This information has the potential to grow over the life of the document. To minimize the chance that this information will overflow the EINOTEPROPS field and to control how much space is required to store this information, see “Restrictions” in the “Creating an External Key Table with Virtual Documents Activities” section in this chapter. 186 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Documents Required Column Data Types For each type of external system the table below shows the data type to use for each of the required columns. Required Field External System Data Types ** Oracle 7 Oracle 8 Sybase (see Note below) DB2 SQL Server Access 2000 EINOTEID*** NUMBER NUMBER This field must not (default to (default to be null zero) zero) INT (default to zero) INTEGER (default to zero) INT (default to zero) NUMBER EIUNID*** CHAR(32) or VARCHAR 2 (32) CHAR(32) or VARCHAR2 (32) CHAR(32) CHAR(32) CHAR(32) TEXT(32) EIMODIFIED*** DATE DATE DATETIME TIMESTAMP DATETIME DATETIME EINOTEPROPS RAW(2K) or LONG RAW (10K) [a, b, c] BLOB (10K) [a, c] IMAGE or binary(255) [a] BLOB(2GB or VARCHAR FOR BIT DATA [a] IMAGE or binary (255) [a] OLE object Note In case-sensitive RDBMS’s such as Sybase, column names must be in upper case. ** For all applications, it is strongly recommended that you limit the number of revision entries maintained for Virtual Documents. In the database advanced properties tab, set the “Limit entries in $Revisions fields” and “Limit entries in $UpdatedBy fields” to three or less. *** It is recommended that the EINOTEID and EIMODIFIED columns be indexed to improve overall performance. Index EIUNID if you use Virtual Documents Activities to support a Domino Web application. [a] — Binary data type size is recommended value. Increase this value for very large forms (for example, more than 100-150 fields) if you anticipate a large number of file attachments or database/document links, or if maximum revision/update history is not set. When in doubt, or if updates fail, select a larger value. Chapter 13: Virtual Documents Activity 187 [b] — Because Oracle 7 limits a user to one long data type per table and does not support BLOBs, you will need to use RAW which has a maximum size of 2KB. A LONG RAW can be used if no other long data types are being used in the table. [c] — If made to work with LONG RAW, there must be no other LONGs within either the data or the key table. Considerations for EINOTEPROPS Of the four control columns required to support Virtual Document Activities, EINOTEID, EIUNID, and EIMODIFIED have a fixed size with specific storage requirements. The EINOTEPROPS column has variable length requirements and requires some special consideration. For most applications, a maximum size of 10KB is sufficient. The EINOTEPROPS column is used for miscellaneous attributes of the Virtual Document which do not need to have separate columns dedicated to them in the external system. Some of the information which is stored in the EINOTEPROPS field has the potential to grow over time and could eventually exceed the EINOTEPROPS field size limit. The size of the information written to the EINOTEPROPS field can be controlled in the Notes database properties box (for the .nsf that you have named in the Activity Document). Open the Advanced properties tab. If the options in the Advanced tab are not visible, have your system administrator change them. The Advanced tab contains options that allow you to limit the amount of data written to the EINOTEPROPS field. These options are “Limit entries in $UpdateBy Fields” and “Limit entries in $Revisions fields.” Lotus Software recommends that you set them to 2 (64 bytes of data) or a maximum of 10 (640 bytes of data). The default value of 0 for these settings means “no limit.” If any of the Notes fields in the form are enabled for encryption using public key encryption, that adds to the amount of data stored in EINOTEPROPS for each encrypted field in a given document. You may need to increase the maximum size for EINOTEPROPS if you are encrypting many fields. See the section entitled “Special Considerations When Using Public Key Encrypted Fields” later in this chapter for more information. 188 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Considerations for EIMODIFIED If changes are made directly to an external system data table (outside of Domino), the EIMODIFIED time stamp should be updated. Doing this enables LEI to determine that the record has changed. This is important in enabling the following features: • Conflict detection — If a user attempts to write a copy of the document which they had been editing when the external system update occurred, they will be notified that a change has been made and be asked if they want to save a conflict document. If the external system EIMODIFIED is not updated, the external system changes would be overwritten. As with any conflict situation, you must evaluate how to resolve it. • Replication — Changes to the table will not be replicated to replicas of the database unless the EIMODIFIED is updated. • Indexing — Under some circumstances, indexes are incrementally updated to include added and changed documents. Views may not display current information until a full index is rebuilt. You can rebuild an index by pressing the Shift F9 keys. Using and Testing the Required External System Fields Before you test or use a Virtual Documents Activity, you should perform the following tasks: 1. Create a form to be monitored by the Virtual Documents Activity. This form must contain the fields that will be mapped to the external system data table columns. 2. Add the four required columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) to the external system data created in step 1 or create an external key table. If using an existing data table — If the data table already contains records, you may be prevented from adding these four columns. If this occurs, create a new table containing the same metadata, add the four required columns, and copy the existing records to the new table. Modify the EINOTEID and set the value to zero. Use SQL if possible. If creating an external key table — When possible, use the Create External Key Table option on the Activity Document, which does not require that you alter the existing data table, and does allow you to “virtualize” your existing data into your specified Domino application. Chapter 13: Virtual Documents Activity 189 3. Create the Virtual Documents Activity by clicking the Add Activity Virtual Documents menu sequence from the LEI Administrator. If you choose to create an external key table, the instructions are very specific and can be found in this document in the section entitled “Creating an External Key Table from the Virtual Documents Activity”. 4. Create a view in the Notes database based on the form being monitored by the Virtual Documents Activity that can display the data fields mapped to the external data. 5. Start the Virtual Documents Activity by clicking either the Start button at the top of the Activity Document or by highlighting the activity name in the LEI Administrator’s Activity View and clicking the Start Activity button. 6. In the LEI Administrator’s Activity View, press the F9 button to ensure that the activity has started without error. Note The activity virtualizes all existing unvirtualized records when you start the activity and before the activity actually become available, with the following caveat. If you disable the Virtualize External System Data option on the Activity Document, this does not occur. 7. The “Activity Started” message indicates that the activity is processing. Check the activity log to see whether the activity has completed. Ensure that the Notes view you are using is designed to display data that corresponds to fields you have mapped to external system data and that the view does not contain a selection formula that excludes the form that the Virtual Documents Activity is monitoring. Data is “virtualized” once at start up and again at specified intervals. This process occurs in the background whenever the Virtual Documents Activity is running. Note Any new data rows that are added by a non-Notes process will be deleted and automatically virtualized in the same manner. You may need to press the F9 button to refresh the view. Once the external system data is virtualized, a message will appear in the activity log and the activity will become accessible. 8. Perform the two tasks below to ensure that you have configured the required tables correctly: The processes of creating, opening, updating, and deleting of Virtual Documents should behave just as in any other native Domino documents. For example, creating a new virtual document using the Notes client results in new rows being inserted into the external system table. Deleting a virtual document may remove the table record. If there is no existing data, enter a record and perform one of these tasks. Note See the section entitled, “Special Considerations When Deleting Virtual Documents” for related information. 190 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Note Virtual Documents respect the security settings of the external system data source. For example, if a user does not have sufficient privileges to create a new row in the external system table being used by the Virtual Documents Activity, he will be unable to save the document. The action will fail with an appropriate error. a. To test the background virtualization manually, add a test row to the external system using , for example, a SQL client. The test row must also use contain and entry for EINOTEID and be set to a zero value. This is most easily tested if the Refresh Frequency option on the Activity Document is set to one minute. Alternatively, you can just stop and restart the activity. b. When the activity starts, it immediately begins to virtualize all unvirtualized external records. The new virtual document, corresponding to the new external records should then appear in the view. Again, you may need to periodically press the F9 key to refresh the view. Once it appears in the view, open and update the document. c. Shut down the activity. Refresh the Notes view. All the virtual documents should disappear from the view. Note Indexing can improve performance. See the section entitled “Indexing to Increase Performance” for more information. Indexing to Increase Performance Indexing is an important requirement for proper Virtual Documents Activity operation, as well as for proper operation of Virtual Attachments when they are used with Virtual Documents Activities. For Virtual Documents, it is strongly recommended that the following external system table columns be indexed for maximum performance. These are some of the additional columns you need to add to the data table, or to an External Key Table: • EINOTEID (always) • EIMODIFIED (always) • EIUNID (if Virtual Documents Activities are used to support a Domino Web application) • Additionally, if you are using the External Key Table, you should also index the data key(s) (the foreign keys) used to map the External Key table to the data table. See the section entitled “Required External System Fields for Virtual Documents Activities” for more information. Chapter 13: Virtual Documents Activity 191 Note When using a Sybase database with Virtual Document Activities, the data table must be indexed. The following matrix shows the columns to index to increase performance in Virtual Documents and in Virtual Attachments: Column Name Required or Optional When to Use EINOTEID R Always EIMODIFIED R Always EIUNID O When supporting Web applications through Domino’s HTTP server. Foreign Key R Always when using external key tables. Special Considerations When Using Public Key Encrypted Fields If any of the fields in a Virtual Document form have been enabled for Public Key Encryption, and a field is subsequently encrypted in a given virtual document, the following behavior and considerations should be noted. • The external data for the publicly encrypted field will be set to NULL in the external table, and will only be visible in the Virtual Document when opened by a user with appropriate privileges. At no time will the data be visible in the external table. The table column which will be mapped to the public key encryption enabled field must be set to allow NULL values and duplicate values, otherwise saving a Virtual Document with encrypted data will fail. • The encrypted data is stored in the EINOTEPROPS column in the external table for the Virtual Document containing encrypted fields. Because of this, you may need to increase the size of EINOTEPROPS beyond the recommended value for this binary column. Generally if the amount of encrypted data is relatively small and of a known quantity this is not an issue. However, if the encrypted fields could potentially hold large arbitrary amounts of text, care should be taken to limit the amount of text in the field, or increase the size of EINOTEPROPS accordingly. For example, a Manager’s Comments text field, as opposed to a employee salary Number field, could become very large if the manager is too verbose. To get a better idea of the amount of encrypted data in a given document, examine the Document Properties Items tab to determine the amount of data encrypted in a typical document. Look for $SealData items and the Data Length of each, where each $SealData item represents an encrypted field in the document. 192 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Special Considerations When Deleting Virtual Documents There are two issues to consider when deleting Virtual Documents; deletion stubs and soft deletions. Deletion Stubs Domino employs a concept called “deletion stubs” when a document is deleted. This means that even though the document data is deleted from the Domino database, a stub of the former document remains in order for other Domino services to function correctly. For example, when replicating a database, the stub identifies the deleted document so that its replica is also deleted in the replica database. When using Virtual Documents, this “deletion stub” is represented in the external system database in one of three ways. • If an external key table is used, the stub is effectively maintained in the key table and the actual data row in the data table is deleted. The only special consideration is that the data key column(s), in the external key table only, should be set to Allow Duplicate Values, even if the corresponding key in the data table is unique. • If you do not use an external key table because you have elected to append the four Domino control columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) directly to the data table, by default the table row is not removed. When a Virtual Document is deleted, it is recognized by Domino as a deleted document and will not be accessible to any Notes user. However, interrogation of the external system table using a SQL client or other external system client will show that the row is in fact not deleted. It remains intact with all the original data. Only the Domino control fields have been changed to designate it as a deletion stub, thus rendering it deleted from Domino’s perspective. • If it is unacceptable to have the external system data still visible through a non-Notes client, disable the Maintain Deletion Stubs option in the Activity Document. However, this action prevents Domino deletions from being seen in database replicas. It may also cause unexpected behavior if replicas of the deleted virtual documents are subsequently updated. Note If you use the Create External Key Table on the Activity Document, you should not disable the Maintain Deletion Stubs option. Chapter 13: Virtual Documents Activity 193 Soft Deletions Domino also supports the notion of soft deletes. If a Notes database has this option enabled, all deletes result in the documents being made invisible except in a specially designed view, analogous to the familiar desktop trash bin. A soft-deleted document can be undeleted at any time. Soft-deleted documents can be set to expire after a certain time interval, when the specified time interval has been reached, the documents are permanently deleted from the Notes database automatically. Virtual Documents Activities completely supports Domino soft deletions. A soft-deleted Virtual Document appears untouched if viewed in the external system through a non-Notes client, but will behave exactly as soft-deleted documents in the Notes database. When the document is permanently deleted it will be deleted from the external system. See the section entitled, “Special Considerations When Deleting Virtual Documents” for more information. Creating a Virtual Documents Activity Using the User Assistant This section describes how to create a Virtual Documents Activity using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” Note You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Documents Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. 194 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 1: Open the Activity Document In either the Activity View or the Favorites View (or a Folder View), click the Add Activity icon in the Action bar. A message box opens that explains the steps the User Assistant will guide you through. If you have not made any selections while the User Assistant is on, clicking the Cancel button on this message box closes the Activity Document and returns you to the LEI Administrator. Once you have made one or more selections while you are in the User Assistant, clicking Cancel causes the User Assistant to shut down, but the Activity Document stays open, in edit mode, displaying the choices you have made up to that point. You can then manually complete the Activity Document or abandon it. Note Any choices you make while using the User Assistant can be changed later by opening the Activity Document in edit mode. Chapter 13: Virtual Documents Activity 195 Step 2: Select the Domino Database A list of Domino databases appears (see illustration below). Select the database that contains the Notes form you want to monitor. Note The database selected must reside on the server where LEI is installed. If you wish to use LEI RealTime functionality in a database that resides on a different Domino server, LEI must be installed on that server. Note If the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the .nsf file you want to use. For more information on browsing and browsing requirements, see “Browsing” in the “LEI Administrator” chapter. Note Real time monitoring uses the Notes replica ID of the application database. A Virtual Documents Activity does not distinguish between the databases that share the same replica id. Databases have the same replica id when they are replicas of each other and when one database is created using the operating system to copy an existing database. 196 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 3: Select the Notes Form to Monitor A list of forms within the selected database appears. These are the forms you will use to display external system data. Select a single form that you want to monitor from the list shown. Chapter 13: Virtual Documents Activity 197 Step 4: Select the Lotus Connection for the Data Source Select the Connection for external data source you want to monitor from those available in the Select Connection dialog box, shown below. When you have selected a connection and there is more than one table in the external database, another database opens asking you to select a table in the external database. See the illustration below. Note When the list of tables is large, you can scroll down the list to find the table you want to select. However, if the list of tables is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” 198 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 13: Virtual Documents Activity 199 Step 5: Map Data Field(s) After specifying the external data source and providing any required connectivity information, such as User Name and Password, the Data Field Mapping dialog box appears. Use this dialog box to map data field(s) between the Domino application and the external data source. Note In general, if there are unmapped columns in the Virtual Documents table, which do not accept NULLs, you will receive a NULL assignment error when creating a new record. The exact wording of this error depends on the Connector you are using. Step 6: Create the External Key Table (optional) If the Notes control fields exist in your external data source, the fields will be mapped automatically. If they do not exist, you must create them using the Create External Key Table function. This procedure is described in the section entitled “Creating an External Key Table” later in this chapter. 200 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 7: Name the Activity Document When prompted, enter a unique name and click OK. After you name the activity and click OK, the following informational message box appears outlining your options. Click OK. The Virtual Documents Activity document appears, showing your selections. The numbers in the document correspond to the major steps the User Assistant uses to create the Virtual Documents Activity. Chapter 13: Virtual Documents Activity 201 Step 8: Select Options Select the specific options that you want for this Virtual Documents Activity. See “Virtual Documents Options,” in this chapter for a full description. Step 9: Select Scheduling Options Select how you want to control the scheduling of the Virtual Documents Activity. See the “Scheduling” section later in this chapter for more information. Step 10: Save and Close the Document Choose File - Save to save the activity. Choose File - Close to close the Virtual Documents Activity document and return to the LEI Administrator. Step 11: Process the Virtual Documents Activity In the LEI Administrator’s Activity View, select the Virtual Documents Activity and then click the Start button to run it. Note You can process the activity using the Start and Stop buttons in the Action bar of an Activities View in the LEI Administrator. When you select “Auto Start” in the Scheduling section of the Activity Document, the Virtual Documents Activity is established each time that the server starts. See the Scheduling section for more information. To optionally stop a Virtual Documents Activity, select the activity name in the Activities view and click the Stop button. Note If you enable the Virtualize External System Data option (default), the activity will first search for and then virtualize all external system table records that have not yet been virtualized. If the table contains many such records, this action can take several minutes or even more depending on table size and network load. Although the LEI Administrator will indicate that the activity has started, while it is virtualizing external system table records, virtual documents not be accessible. Check the activity log progress (periodically refresh your view by pressing the F9 key) and results of this virtualization process. When it is complete, a message will appear and the activity will become accessible. 202 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Creating a Virtual Documents Activity without the User Assistant You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Documents Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. If you disable the User Assistant, a blank Activity Document is opened for you to define the activity. This section describes how to create a Virtual Documents Activity without using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” Step 1: Open the Activity Document Click the Add Activity icon. The Virtual Documents Activity Document appears. Step 2: Enter a Name for the Activity Enter a unique name for the activity in the Name field at the top of the document. Step 3: Select the Domino Application and the Form to Monitor Click the down-arrow icon in the Domino application section. A list of databases appears. Select the database that contains the Domino application that you want to monitor. You are also prompted for a form name. Note When the list is large, you can scroll down the list to find the Domino application (nsf) you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the .nsf file you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Note Real time monitoring uses the Notes replica id of the application database. A Virtual Documents Activity does not distinguish between the databases that share the same replica id. Databases have the same replica id when they are replicas of each other and when one database is created using the operating system to copy an existing database. Chapter 13: Virtual Documents Activity 203 Step 4: Select the Lotus Connection for the Data Source Click the down-arrow icon in the External System section to select from existing Lotus Connection documents. Note When the list of tables is large, you can scroll down the list to find the table you want to select. However, if the list of tables is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Step 5: Mapping Click the down-arrow icon in the Mapping section. The field mapping dialog box appears. Use this dialog box to map the data fields between the Domino application and the external data source. Note If there are unmapped columns in the Virtual Documents table, which do not accept NULLs, you will receive a NULL assignment error when creating a new record. The exact wording of this error depends on the Connector you are using. Step 6: Create the External Key Table (optional) If the Notes control fields exist in your external data source, the fields will be mapped automatically. If they do not exist, you must create them using the Create External Key Table function. This procedure is described in the section entitled “Creating an External Key Table” later in this chapter. Step 7: Select Options and Event Options Select the options and the event options you want on the “Options” and “Event Options” tabs in the document. See the section “Virtual Documents Activity Options and Event Options” later in this chapter for more information. Step 8: Select Scheduling Options Select how you want to control the scheduling of the Virtual Documents Activity. The options are Manual, AutoStart, and Custom. See the “Scheduling” section later in this chapter for more information. Step 9: Save and Close the Document Choose File - Save to save the Activity Document. Choose File - Close to close the Activity Document and return to the LEI Administrator. 204 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 10: Process the Virtual Documents Activity In the LEI Administrator’s Activity View, select the Virtual Documents Activity and then click the Start button to run it. Note You can process the activity using the Start and Stop buttons in the Action bar of an Activities View in the LEI Administrator. When you select “Auto Start” in the Scheduling section of the Activity Document, the Virtual Documents Activity is established each time that the server starts. See the Scheduling section for more information. To optionally stop a Virtual Documents Activity, select the activity name in the Activities view and click the Stop button. Note If you enable the Virtualize External System Data option (default), the activity will first search for and then virtualize all external system table records that have not yet been virtualized. If the table contains many such records, this action can take several minutes or even more depending on table size and network load. Although the LEI Administrator will indicate that the activity has started, while it is virtualizing external system table records, virtual documents not be accessible. Check the activity log progress (periodically refresh your view by pressing the F9 key) and results of this virtualization process. When it is complete, a message will appear and the activity will become accessible. Virtual Documents Activity Document Options Once you have specified the Domino application, the connection you want to use, and set up mapping, there are a number of options for you to consider in setting up the activity. The options are broken down into two areas: Options and Event Options. • The Options tab contains the following three areas. • General Options • Integrated Credentials • Virtual Attachments • The Event Options tab contains the following four areas. • Create • Open • Update • Delete Chapter 13: Virtual Documents Activity 205 See the chapter entitled “Introduction to LEI Activities” for a description of the following sections that are common to all Activity Documents: • Activity Name • Scheduling • Category • Comments Virtual Documents Options Tab The Options tab contains three tabs — General Options, Integrated Credentials, and Virtual Attachments. General Options The General Options section provides options that can be applied to the Virtual Documents Activity regardless of the type of event that the activity monitors. Each of the General Options is described after the illustration. Note To use Virtual Documents you must set up four fields in the external data source. See “Required External System Fields for Virtual Documents Activities” earlier in this chapter for details. 206 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options Description Key Table Option Select either of two options: Keys Are in Connector Table — If you select this option, the key fields must be present in the external database table. Use External Key Table — Select this option to use virtual view facilities without modifying the external system database table. The required Notes fields are stored in an external table that includes key fields that are used to match the rows with the data table. If you want to create an external key table, you must select this option. When you select this option, two additional choices become available: “Key Table Name” and “Key Fields”, both of which must also be specified in order to create an external key table. w Key Table Name — The name of the external table which contains the required fields to synchronize with the Domino database. This is a separate table from the Data Source table. If using this field in conjunction with creating an external key table, enter a new name here - not the name of an existing table. w Keys Field(s) — Fields that map to unique records in the data table. Note You cannot run a Virtual Documents Activity, using the External Key Table option, if the connection is to ODBC using Data Direct 3.5 Lotus-branded drivers. Note See the section entitled “Creating an External Key Table” for complete step-by-step instructions. Maintain Deletion Stubs If selected (default), the external data row will not be removed but will only be internally marked as deleted and will no longer be accessible as a Virtual Document in Notes. These rows may be periodically purged and removed if purging is enabled for the Notes databases. If disabled, the external data row will always be removed on a Virtual Document delete. However, this deletion will not be replicated to other databases by Domino. continued Chapter 13: Virtual Documents Activity 207 General Options Description Virtualize External System Data Select Virtualize External System Data to enable this activity to automatically virtualize external database system data when the activity starts and at periodic intervals where new data is added through an external (non-Notes) client. Note Only one Virtual Document Activity should have this option enabled per external data table. When you select the Virtualize External System Data option, the Refresh Frequency option becomes available. w Refresh Frequency: Frequency that new records from non-Notes sources are inserted in the view. The choices are minutes, hours, and days. The default is once every hour. Max. Connections This option sets the maximum number of connections to the external database that can be open to service concurrent user requests simultaneously. The minimum number of connections for Virtual Documents is 2. If you experience significant delays, you can increase this number. LEI opens one connection to the external data source when the first Domino application event occurs. If two or more events occur simultaneously, additional connections are made, up to the maximum number of connections specified by this option. When the maximum number of connections is reached, subsequent events are queued and occur when each preceding event is serviced. Each connection lasts only as long as necessary to read from or write data to the external data source. While the connection is persistent, the time required to service each event is minimal and depends on the amount of data being read or written. The maximum number of connections, therefore, does not need to be very large to service multiple events. Data Integrity Prevent data loss — Write an error to the log for any data loss resulting from a data type conversion. Allow precision loss — Do not report loss of numerical or DATETIME precision as a result of a data type conversion. This allows some loss of precision without stopping the transfer. This option is the default. Allow precision loss & truncation of text — Enable this option to have LEI allow precision loss as well as truncation of text data when necessary to conform to field lengths in the external database. continued 208 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options Description Trim Trailing Spaces This option affects any trailing spaces that exist in the Text fields of the external data. Trim spaces on all fields — Select this setting to trim trailing spaces only from data fields, not from the key fields. (Default) Do not trim spaces on any fields — Select this option to leave trailing spaces in all fields. Trailing spaces may be required to ensure matching of fields between Notes and the external data. Note When data is put into external system fields of fixed length and then retrieved, the connection with the external system may be lost because the external system has padded the data with spaces to fit the fixed length of the field. If this occurs, use either a variable length field (VARCHAR) in the external system database, or enable the “Trim spaces on all non-key fields” option. New Line Delimiter for Connection Select the option you want to use to convert the Notes new line delimiter to a new line delimiter in another system. Creating an External Key Table This section describes how to create and use an external table for the required additional external system columns. It also describes the restrictions involved in this approach. Although modifying an existing data table, or creating a data table with the additional needed columns, is the most logical way to enable Virtual Documents to access external system data, it may not always be desirable. Using an External Key Table eliminates the need to modify the existing data table but also requires additional steps. If you choose to store these values in a separate external key table, use this procedure. The Virtual Documents Activity requires that the data source contain either a dedicated table or fields in an existing table to store Notes control information. Since no information about the document is stored in Notes, there needs to be a place to store the document universal ID, creation date/time, and other system values. Chapter 13: Virtual Documents Activity 209 The following steps describe how to create an external key table using a Virtual Documents Activity. 1. Determine whether you will create your data table or use an existing one. The data table must contain one or more columns that form a unique key set for each row of data. More than one column can be used to achieve unique values. As with the method where you add the four required columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) to an existing data table, an external key table must also contain these columns, in addition to at least one key column from the data table. The key column(s) allows the external key table to mesh with the same key columns in the data table. The key column(s) should have the same name and data type as the counterpart(s) in the data table. The Activity Document itself helps aid you in specifying these required conditions. 2. Create a new Virtual Documents Activity using the Add Activity button in the LEI Administrator action bar. See either “Creating a Virtual Documents Activity Using the User Assistant” or “Creating a Virtual Documents Activity Without the User Assistant” later in this chapter for complete instructions. 3. Click the “Options” tab on the “General Options” tab in the middle of the Activity Document. 210 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 4. Select the Key Table Option “Use External Key Table” check box. As a result, the “Key Table Name” and “Key Field(s)” option appear. You must enter data in these fields now. 5. In the “Key table name” field, enter a table name now. 6. In the “Key field(s)” field, choose at least one key column name(s) from the resultant list. To test the required fields, see the section entitled “Using and Testing the Required Tables” for more information. Note Certain fields should be indexed at the external system to improve performance. See the section entitled “Indexing to Increase Performance” for more information. 7. Select the Create External Key Table button that appears just above the Key Table Option item on the Activity Document. 8. The Create Key Table dialog appears, enabling you to select the options that will set an optimal size limit for some fields and create indexes as appropriate for others. Complete the form using the descriptions presented below. Chapter 13: Virtual Documents Activity 211 The bottom part of the dialog contains questions to help configure the key table appropriately for your environment. These questions vary depending on which Lotus Connector is being used for the activity. Note If the user ID and password stored in the Connection Document don’t have sufficient access to create tables in your destination database, uncheck the “Use Connector Credentials” option. This lets you enter a different user ID and password that will only be used for this one operation. This login information will not be stored in the LEI Administrator database. The Create Key Table dialog box contains the following fields: • Table Name — This field displays the table name that you entered in the Key Table Name field in the General Options section of the Activity Document. Note You must include the owner name in the first part of the Table Name entry. For example, to create a table named EMPTRANSFER with an owner name of DB2ADMIN, you would enter DB2ADMIN.EMPTRANSFER in the Table Name field on the Create Attachment Table form. • Use Connector Credentials — You need a relational database user ID with privilege to create metadata. If the user ID in the Connection Document does not have this privilege, uncheck this box and enter a different user ID and password. • Optimization Questions — Depending on the type of Lotus connection that you specify on your Virtual Documents Activity Document, optimization questions will appear in your dialog box. However, when connecting to Oracle 8, DB2, or OLE DB, it is not necessary to specify the size of the system field area (EINOTEPROPS). In such instances, the optimization questions and “System field storage … bytes” field will not be present in the dialog box. These questions are used to calculate an optimal size for the EINOTEPROPS field and to decide which fields should be indexed for best performance. Depending on your answers, LEI will calculate a suggested value for the field size limit. However, you may override this suggestion by entering a different value in the “System field storage … bytes” field in this dialog. If you leave this field blank, the “suggested” value will be used. 212 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • System field storage — Use this to override the suggested size for the EINOTEPROPS field. See the description for Optimization Questions above for more information. 9. When you have completed the form using the dialog box descriptions above, click OK in the upper right of the form to create the key table. 10. A message appears, indicating success. For information about using these fields as stored procedure arguments, see the section entitled “Using Stored Procedures in Virtual Documents Events.” For information about which fields will be indexed by the table creation tool, refer to the section entitled “Usage Requirements for Virtual Documents and Virtual Attachments.” Considerations When creating Virtual Documents with the External Key Table option, the following considerations apply: Chapter 13: Virtual Documents Activity 213 • You cannot run a Virtual Documents Activity using the External Key Table option if the connection is to ODBC using Data Direct 3.5 Lotus-branded drivers. • You must enable writeback in order to modify a key using an external key table. Modification of non-key fields does not require writeback. Alternatively, you can delete the record and then recreate it with a new key value. • The data table and the External Key Table must reside on the same external system data source (same connector/server/database/userid/password, different table name). • If rows are deleted from the data table by another application and the corresponding entry in the ID table is not deleted, you will receive “unable to retrieve external record” errors when you attempt to access that Virtual Document. See the section entitled, “Special Considerations When Deleting Virtual Attachments” for more information. • If there are data rows with duplicate keys, only one ID table entry will be created, only the first data row will be retrieved on open, and both records will be modified on delete and update. • For maximum performance, always index the data table key columns. See the section entitled “Indexing to Increase Performance” for more information. • Regardless of whether the corresponding key columns in the data table may be constrained to not allow duplicate values, do not impose that setting on the key columns in the external key table. In the external key table, key columns should be set to allow duplicate values. • There is no support currently for adding the four required columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) to an existing table from the Activity Document. Most external data sources do not permit us to do this. If you would prefer to add the fields to an existing table, you might create the external table first, then use your relational database configuration tool to copy fields to the table you want them to appear in. Notice which columns are indexed in the external key table and retain those indexes when you copy the fields. Alternatively, you can create a new table with the new columns and then use a Direct Transfer Activity to copy the data from the old table to the new table. 214 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Integrated Credentials User credentials for the external system are looked up when the Integrated Credentials options is selected. The credentials are used to connect to the external system. This increases Domino security by integrating it with the security already in place for the external system. Note The integrated credentials database is described in the “Introduction to Activities” chapter of this document; see the section entitled “Integrated Credentials Database” for information about creating and using an integrated credentials database. Note If you are using integrated credentials with Virtual Documents through an HTTP client, you also need a server credential document. Note When using integrated credentials with an HTTP client, if the .nsf does not prompt for a user name and password, the user is logged in as “Anonymous.” As a result, a credential entry for “Anonymous” may be needed. Each of the Integrated Credentials options is described after the illustration. Chapter 13: Virtual Documents Activity 215 Integrated Credentials Options Description Use Connector Credentials Use the user ID and password that was entered into the Connection Document you are using in this activity. Lookup Credentials When you select this option the form expands to include the following options: Missing Credentials — In case of a credentials lookup failure, either the operation can fail (this is the default action) or default credentials in the activity can be used. Credentials DB Filepath — The name of the credentials database to be used for storing Notes user names and corresponding external systems credentials. The path is specified relative to the LEI directory (for example leidir). Note You must have created this database using the template leicred.ntf. See “Setting Up and Using Integrated Credentials” in the Introduction to Activities chapter of this document. Connection Cache Size — Maximum number of distinct credentials active at one time. Note If the credentials used to access the external database system do not have write privileges to the tables, Notes users that attempt to modify (delete, update, create) documents will receive an error. Integrated Credentials and Full Text Indexing In order to perform full text indexing with Virtual Documents or Virtual Fields in conjunction with integrated credentials, you must add the server ID to the integrated credentials database. Integrated Credentials and Server to Server Replication In order to perform server to server replication in conjunction with integrated credentials, you must add the server ID of the server being replicated with to the integrated credentials database. For example, if you want to replicate a Virtual Documents or Virtual Fields-enabled database on Server A to Server B, you must add Server B’s server ID to the integrated credentials database on Server A. 216 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Integrated Credentials in a Domino Cluster The server IDs for all the servers in a Domino cluster that contain replicas of the Virtual Documents or Virtual Fields-enabled database must also be added to the integrated credential database. For example, if a Virtual Documents-enabled database on Server A has replicas on Servers B and C, and Servers A, B, and C are part of the same Domino cluster, the server IDs for Servers B and C must be added to the integrated credentials database on Server A. If this is not done, the cluster replication process will fail to keep the replicas synchronized. Virtual Attachments The Virtual Attachments feature allows you to use Notes to add attachments to the external system database. File attachments will be stored in the external system rather than in your Domino application. Once added, they can be accessed through the virtual views. Note Storing the file name and Notes ID makes identification and retrieval of the attachments easier but requires additional columns in the external system table. The list of required and optional Virtual Attachment columns is as follows: Column Name Required or Optional EIDBID Required EIFILEID Required EIFILESIZE Required EIUNID Optional EIFILENAME Optional EICONTENTS Required You do not need to include the EIUNID or EIFILENAME fields in the attachments table. They are not needed by LEI, but LEI maintains them if they are provided. These fields allow external database applications to more easily recognize and access the data outside of LEI/Domino. EIUNID is the noteid value that corresponds to the EIUNID field in the row of the data/key table. EIFILENAME is the name of the attachment as specified in the note. You can control this functionality using the Store Optional Fields option on the Activity Document. An important restriction is that the virtual attachments table must be in the same external system and in the same database as the external system data being accessed by the activity. The virtual attachments table cannot reside in a different external system or database than the one being used by the Virtual Documents Activity. This is because the same Connection Document is used to access the external system. Chapter 13: Virtual Documents Activity 217 The Virtual Attachments feature allows you to add attachments to virtual documents as you would to a native Notes document. The exception is that file attachments are stored in the external system rather than in your Domino application. The attachment data is stored in a binary field in an external system table that you create. Other columns in the external system table store key data, which enables the activity to locate a specific attachment. A single table can be used for multiple Virtual Documents Activities. The following considerations are helpful as you use this feature. • You must select the Virtualize Attachments option when you use attachments in a Virtual Document. You cannot save an attachment in a Virtual Documents Activity unless it is virtualized. If you do not, your attachments will not be saved. • If the “fixup” utility is run on the Notes database while a monitored document is open, you must either close and reopen document or save the document to access any restored Virtual Attachments. For information about “fixup” see the Domino server documentation. Virtual Attachments Options Description Attachments Virtualize Attachments — You must select this option to be able to store attachments with your virtual documents. Enabling this option requires that you create and specify an external system table to contain the contents of the attachment. Note You must specify an attachment table in order to create a virtual attachments table. Attachment table (Required if “Attachments” is selected.) Specify a separate table for the contents of the attachments. In addition to the contents of the attachments, this location also contains a locator key. Optionally, the file name and Notes ID may be stored as an additional data element. Select “Additional data” for this option. Note When you create a virtual attachment table in Sybase, the column names must be in upper case. Note See the section entitled “Creating a Virtual Attachments Table” for complete step-by-step instructions. Additional data Select this option to add the attachment file name and its Notes ID to the attachment table. This option is not required. Note Storing the file name and Notes ID makes identification and retrieval of the attachments easier but requires additional columns in the external system table. 218 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Indexing Fields for Increased Performance If you are using Virtual Attachments, the Virtual Attachment Table should be indexed on EIFILEID. In addition, the Virtual Attachment Table EIDBID column should be indexed if the table supports multiple databases. It is used by multiple Virtual Documents activities, some of which monitor forms from different Domino databases. Fields and Data Types The following table describes the necessary fields and data types that are required for the attachment table at the external system as well as the fields that are required on the Notes form. Attachment Table Fields External System Data Types Oracle Informix Sybase [a] DB2 [b] EIDBID** (required) VARCHAR2 (16) or CHAR (16) CHAR (16) CHAR (16) CHARACTER CHAR (16) (32) TEXT (16) EIFILEID** (required) NUMBER INT INT INTEGER INT NUMBER INTEGER EIFILESIZE [c] NUMBER (8) (required) INT INT INTEGER INT NUMBER LONG INTEGER EIUNID [d] (optional) CHAR (32) or VARCHAR2 (32) CHAR (32) CHAR (32) CHAR (32) CHAR (32) TEXT (32) EIFILENAME (optional) VARCHAR2 (255) VARCHAR VARCHAR (255) (255) VARCHAR (255) VARCHAR (255) TEXT (255) BYTE BLOB (1048576) IMAGE OLE Object EICONTENTS LONG RAW required) IMAGE SQL Server Access 2000 ** Index these fields at the external system to increase performance. a) In case sensitive RDBMS’s, such as Sybase, column names must be in upper case, for example, EIUNID. b) DB2 — with up to a 1MB attachment. c) EIFILESIZE column length may be adjusted, but it must remain an integer (X,0). Chapter 13: Virtual Documents Activity 219 d) EIUNID is a DWORD, or an unsigned 32-bit integer. According to the formula 2^n - 1, this gives a maximum EIUNID of 4,294,967,295 which requires 10 digits. Specifying the Attachment Table and Schema By default, if a schema is not specified for the virtual attachment table, the user making the connection is used by default. For example, if the user is TESTONE and the schema is DBSOURCE, the user will be taken as the schema, TESTONE.MY_TABLE. If the virtual attachment table is picked from the selection dialog, the schema name is added as a prefix. However, if the user types the table name in by hand and there is a difference between the schema and the connection user, he must type the schema name as a prefix, for example, DBSOURCE.MY_TABLE. Any attachments that were not entered into the external system through the Virtual Attachment process will not be accessible through either Virtual Fields or Virtual Documents. For example, if you added attachments to the external system database through another process, for example SQL commands, you cannot access them through Virtual Fields or Virtual Documents. You can create the table using SQL commands. Creating a Virtual Attachments Table In the Virtual Documents and Virtual Fields Activity Documents, you can configure the activity to use a relational table to store file attachments as binary objects by creating a virtual attachments table. 1. Click the “Virtual Attachments” tab in the middle of the Activity Document. 220 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 2. Click the Attachments: Virtualize Attachments option. 3. In the Attachment table field, enter the name of the table that you want to create. 4. Optionally specify that you wish to store optional fields. 5. Click the Create Virtual Attachments Table button. Chapter 13: Virtual Documents Activity 221 6. The Create Attachment Table dialog appears, enabling you to select the options that will set an optimal size limit for some fields and create indexes as appropriate for others. Complete the form using the descriptions presented below. • Table Name — This field displays the table name that you entered in the “Attachment table name” field of the Activity Document. Note You must include the owner name in the first part of the Table Name entry. For example, to create a table named EMPTRANSFER with an owner name of DB2ADMIN, you would enter DB2ADMIN.EMPTRANSFER in the Table Name field on the Create Attachment Table form. • Use Connector Credentials — You need a relational database user ID with privilege to create metadata. If the user ID in the Connection Document does not have this privilege, uncheck this box and enter a different user ID and password. 7. When you have completed the form using the dialog box descriptions above, click OK in the upper right of the form to continue. 8. A message appears, indicating success. Note If the table already exists, you will prompted whether to overwrite it. If you answer Yes, the existing table and all the data in it are erased. Since most connection types don’t allow existing tables to have fields added or field properties adjusted by an external program, 222 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide changing existing attachment tables that were incorrectly created is not supported. For information about which fields will be indexed by the table creation tool, see the section in this chapter entitled “Virtual Attachments in Virtual Documents Activities.” Event Options Tab The Create, Open, Update, and Delete options are available from the Event Options tab. They provide many options that depend on the type of event the Virtual Documents Activity monitors. The monitored events occur in the Notes form that you specified in this Activity Document. Each of the options is described in the tables on the following pages. Unlike Virtual Fields activities, Virtual Documents Activities always process these four basic form events. Create Options When monitoring Document Create Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Create Options Description Pre-Create Formula Executes a Notes formula language statement before the document data is stored. You can use this option to modify existing fields or to compute values for additional fields. For example: FIELD LASTNAME:=@if(LASTNAME = ““;“NA“;LASTNAME);”” Note If you are connecting to an Oracle external system and you must use a text field(s) of fixed length as a key field(s), use a formula filter to pad the specific field to the correct length. See the Lotus Connector for Oracle 7 or 8 information in Lotus Connectors and Connectivity Guide for more information. continued Chapter 13: Virtual Documents Activity 223 Create Options Description Stored Procedure Executes a stored procedure in the external data source to store data that has been entered into the Notes document. The field(s) are supplied to the stored procedure as input parameters. To specify the stored procedure and see the parameters that will be passed to the stored procedure and their order, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Domino/LEI supports stored procedure browsing. In a Virtual Documents Activity, the required parameter EINOTEID, one of the four required columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) also appears. Note When you use stored procedures with Virtual Documents, you must add the EINOTEID parameters to the stored procedure at the external system. See the section “Using Stored Procedures in Virtual Documents Events” for more information. Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. Stored Procedure Output Enable Procedure Output — This option is available only if a stored procedure has been specified in the “Stored Procedure” field. When this option is selected, it opens up the “Stored Procedure Output Mapping” section of the Activity document, where you enter input and output field names. This option enables the return of data from a stored procedure in the external system to the Domino application (that is, the Notes document). The Activity’s key(s) and field(s) are supplied to the stored procedure as input parameters. To see the arguments that will be passed to the stored procedure, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Note Only the Lotus Connector for DB2 supports output parameters. Stored Procedure Output Mapping Domino Application — Specifies the fields in the Domino application (Notes document) that will receive the stored procedure output. Lotus Connection — Specifies the fields from the external system that contain the stored procedure output. The first field from the external system maps to the first destination field, the second external system field to the second destination field, and so on. Note These fields must be specified in the “Mapping” section of the Activity document. 224 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Open Options When monitoring Document Open Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Open Options Description Post-Open Formula Executes a Notes formula immediately after data is retrieved from the external system. You can use this option to compute additional fields or modify existing fields. For example: FIELD FIRSTNAME := @if(FIRSTNAME = “Al“;“Albert“;FIRSTNAME);”” Note Do not use Post-Open Formulas when using conflict detection. Stored Procedure Executes a stored procedure in the external data source to determine the data that will be retrieved into the Notes document. To see the arguments that will be passed to the stored procedure, type the name of the stored procedure in this field or select the stored procedure from the drop down list and then press F9. In the case of Virtual Documents of the four required fields, only the EINOTEID field is necessary as an input field. The rest of the fields are output fields. These are specified according to the external system. For Connectors that support procedure output parameters, the key fields are passed as input/output parameters and the other fields are passed as output parameters. Note Domino/LEI supports stored procedure browsing. Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. Stored Procedure Output Mapping These fields are visible only if “Stored Procedure Output” in either a Create or an Update event has been selected, see either Create or Update event for more information. Chapter 13: Virtual Documents Activity 225 Update Options When monitoring Document Update Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Update Options Description Pre-Update Formula When a Notes document is updated, this option executes a Notes formula statement before the data is saved in the external database. Stored Procedure When a document is updated, you can use this option to execute a Notes formula before the data is saved in the external system. The field(s) are supplied to the stored procedure as input parameters. To specify the stored procedure and see the parameters that will be passed to the stored procedure and their order, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Domino/LEI supports stored procedure browsing. In a Virtual Documents Activity, the required parameter EINOTEID, one of the four required columns (EINOTEID, EIUNID, EIMODIFIED, and EINOTEPROPS) also appears. Note See the “LEI and Data” appendix for error situations you may encounter using Sybase or OLE DB stored procedures that return a result set. continued 226 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Update Options Description Stored Procedure Output Enable Procedure Output — This option is available only if a stored procedure has been specified in the “Stored Procedure” field. When this option is selected, it opens the “Stored Procedure Output Mapping” section, where you enter input and output field names. This option enables the return of data from a stored procedure in the external system to the Domino application (Notes document). The activity’s key(s) and field(s) are supplied to the stored procedure as input parameters. To see the arguments that will be passed to the stored procedure, type the stored procedure name in this field and then press F9. The fields that will be passed to the stored procedure are displayed next to the stored procedure name. Note Only the Lotus Connector for DB2 supports output parameters. Conflict Detection Check External Data for Changes — Ensures that the external data has not changed since the Notes document was opened. If it has changed, an update to the external data source will fail. If this option is enabled and if you make changes to data in a Notes document and then save the document, you must exit the document before making any more changes — even though you have saved the document. Note When using an HTTP server to access documents, “Conflict Detection” is not supported. Field Level Updates Only Update Changed Fields — Causes the Virtual Documents Activity to not update fields in the external data source unless the corresponding fields in the Notes document have been edited. Use this option if you have triggers in the external system that monitor updates to individual columns of a table. Stored Procedure Output Mapping Domino Application — Specify the fields in the Domino application (Notes document) that will receive the stored procedure output. Lotus Connection — Specify the fields from the external system that contain the stored procedure output. The first field from the external system maps to the first destination field, the second external system field to the second destination field, and so on. Note These fields must be specified in the “Mapping” section of the activity. Chapter 13: Virtual Documents Activity 227 Delete Options When monitoring Document deletion Events, several options are available depending on the choices you make. A description of each of the options follows the illustration. Delete Options Description Pre-Delete Formula Notes formula language statement to execute on the Notes document immediately prior to the deletion in the external data source. Here is an example that sends an E-mail containing information about a deleted document: FIELD LASTNAME:=LASTNAME; FIELD FIRSTNAME:=FIRSTNAME; @MailSend(“John Q Public“;““;““;“Doc Deleted“;“Doc deleted from MyDatabase w/ Firstname/Lastname of:“;FIRSTNAME:LASTNAME);”” Note It may be necessary to specify that activity data be saved in the Notes document; otherwise, the specified formula may access the field values after they’ve been deleted. If this were the case for the example above, FIRSTNAME and LASTNAME would be empty. To specify that activity data be saved, display the Data Storage tab under the General Options tab, and select the “Leave all/selected realtime fields in document” option. continued 228 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Delete Options Description Stored Procedure When a document is deleted this option executes a stored procedure in the external data source to remove the data related to the document. To see the arguments that will be passed to the stored procedure, type the name of the stored procedure in this field or select the stored procedure from the drop down list and then press F9. Domino/LEI supports stored procedure browsing. In the case of Virtual Documents of the four required fields, only the EINOTEID field is necessary as an input field. The rest of the fields are output fields. These are specified according to the external system. For Connectors that support procedure output parameters, the key fields are passed as input/output parameters and the other fields are passed as output parameters. Note When you use stored procedures with Virtual Documents, you must add additional parameters to the stored procedure. See the section “Using Stored Procedures in Virtual Documents Events” for more information. Note See the “LEI and Data” appendix for error situations you may encounter using stored procedures that return a result set with Sybase or OLE DB. Stored Procedure Output Mapping These fields are visible only if “Stored Procedure Output” in either a Create or an Update event has been selected. See either Create or Update event for more information. Using Stored Procedures in Virtual Documents Events Any of the four basic events handled by the Virtual Documents Activity (Create, Open, Update, Delete) can, instead, be handled through a stored procedure defined in the external system. These procedures can add specific external logic to the operation, such as modifying data fields, or add reporting functionality for example. However, if you use a stored procedure, you must follow certain important guidelines for writing the stored procedure. Chapter 13: Virtual Documents Activity 229 Currently you cannot use an external key table with stored procedures. First, EINOTEID must always be one of the parameters to the stored procedure. This parameter corresponds to one of the additional columns required in the external system table by the Virtual Document Activity. The EINOTEID parameter must be used as the unique key to access the external system table from within the stored procedure. See the section “Required External System Fields for Virtual Document Activities” for more information about this field and the data type it requires. Second, for all the events except the delete event, ALL the external system table data fields mapped in the Virtual Document Activity must be present as parameters. No other parameters are allowed. For a delete event stored procedure, only the EINOTEID is required. Third, the stored procedure must perform an operation consistent with the Virtual Document event to which it is associated. For example, an Open stored procedure must use the EINOTEID as a key to perform a Select operation. A Create stored procedure must insert a new row into the table. The procedure may alter data, access other tables, and perform other operations, but the operation on the external system table specified in the Virtual Documents Activity must be consistent with the event. Note The EINOTEID parameter value must never be altered in any way. Finally, special care must be taken with Delete stored procedures. If the stored procedure elects to actually remove the external system table row, using the EINOTEID as the key, Domino will lose the ability to replicate the Virtual Document deletion to replica databases. If replicating Virtual Document deletions is required, do not allow the stored procedure to actually remove the row. Subsequent internal operations will mark the row as deleted to Domino, but will leave it intact in the table. You may still use a delete stored procedure to simply perform a reporting operation or some other housekeeping operation for example. If you only wish to use the delete stored procedure to remove the external system row, you can instead simply disable the “Maintain Deletion Stubs” option in the Virtual Documents Activity. The following summarizes the stored procedure parameter requirements: For Create or Update events, in addition to the mapped data fields, stored procedures must accept the EINOTEID field as an input parameter. 230 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide For Open events, in addition to the mapped data fields, stored procedures must accept the EINOTEID field as an input/output parameter. For Delete events, only the EINOTEID must be mapped as an input parameter. There is nothing you have to do to ensure that EINOTEID is being mapped other than be certain that the name and data type is correct. Stored Procedure Examples Here are some simple DB2 stored procedure examples that illustrate the use of the EINOTEID parameter. These examples assume that the Virtual Document Activity has only two mapped data fields: EMPNO and SALARY. OPEN Example Here is a DB2 stored procedure open example: CREATE PROCEDURE MRL_EMP_EIVD_OPEN (INOUT EINOTEID INT, OUT EMPNO INT, OUT SALARY INT) LANGUAGE SQL ---------------------------------------------------------------------- SQL Stored Procedure --------------------------------------------------------------------P1: BEGIN DECLARE KeyVar INT; SET KeyVar = EINOTEID; SELECT EMPNO, (SALARY * 2) AS SALARY, EINOTEID INTO EMPNO, SALARY FROM MRL_EMP_EIVD WHERE MRL_EMP_EIVD.EINOTEID = KeyVar; END P1 Chapter 13: Virtual Documents Activity 231 UPDATE Example Here is a DB2 stored procedure update example: CREATE PROCEDURE MRL_EMP_EIVD_UPDATE EMPNO INT, IN SALARY INT) (IN EINOTEID INT, IN LANGUAGE SQL ------------------------------------------------------------------------ SQL Stored Procedure ----------------------------------------------------------------------P1: BEGIN DECLARE KeyVar INT; DECLARE VarEmpno INT; DECLARE VarSalary INT; SET KeyVar = EINOTEID; SET VarEmpno = EMPNO; SET VarSalary = SALARY; UPDATE MRL_EMP_EIVD SET EMPNO=VarEmpno, SALARY=VarSalary*2 WHERE MRL_EMP_EIVD.EINOTEID = KeyVar; END P1 UPDATE Example (with output parameters) Here is a DB2 stored procedure update example that also has output parameters: CREATE PROCEDURE MRL_EMP_EIVD_UPDATE_OUTPARAM INT, INOUT EMPNO INT, INOUT SALARY INT) (IN EINOTEID LANGUAGE SQL ------------------------------------------------------------------------ SQL Stored Procedure ----------------------------------------------------------------------P1: BEGIN DECLARE KeyVar INT; 232 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide DECLARE VarEmpno INT; DECLARE VarSalary INT; SET KeyVar = EINOTEID; SET VarEmpno = EMPNO; SET VarSalary = SALARY; UPDATE MRL_EMP_EIVD SET EMPNO=VarEmpno, SALARY=VarSalary*2 WHERE MRL_EMP_EIVD.EINOTEID = KeyVar; SET EMPNO = VarEmpno; SET SALARY = VarSalary*2; END P1 CREATE Example Here is a DB2 stored procedure create example: CREATE PROCEDURE MRL_EMP_EIVD_CREATE_OUTPARAM INT, IN EMPNO INT, IN SALARY INT) (IN EINOTEID LANGUAGE SQL ------------------------------------------------------------------------ SQL Stored Procedure ----------------------------------------------------------------------P1: BEGIN DECLARE KeyVar INT; DECLARE VarEmpno INT; DECLARE VarSalary INT; SET KeyVar = EINOTEID; SET VarEmpno = EMPNO; SET VarSalary = SALARY * 2; INSERT INTO MRL_EMP_EIVD (EINOTEID, EMPNO, SALARY) VALUES (KeyVar, VarEmpno, VarSalary); END P1 Chapter 13: Virtual Documents Activity 233 CREATE Example (with output parameters) Here is a DB2 stored procedure create example that also has output parameters: CREATE PROCEDURE MRL_EMP_EIVD_CREATE_OUTPARAM INT, INOUT EMPNO INT, INOUT SALARY INT) (IN EINOTEID LANGUAGE SQL ------------------------------------------------------------------------ SQL Stored Procedure ----------------------------------------------------------------------P1: BEGIN DECLARE KeyVar INT; DECLARE VarEmpno INT; DECLARE VarSalary INT; SET KeyVar = EINOTEID; SET VarEmpno = EMPNO; SET VarSalary = SALARY * 2; INSERT INTO MRL_EMP_EIVD (EINOTEID, EMPNO, SALARY) VALUES (KeyVar, VarEmpno, VarSalary); SET EMPNO = VarEmpno; SET SALARY = VarSalary; END P1 DELETE Example Here is a DB2 stored procedure open example: CREATE PROCEDURE MRL_EMP_EIVD_DELETE (IN EINOTEID INT) LANGUAGE SQL ------------------------------------------------------------------------ SQL Stored Procedure ----------------------------------------------------------------------P1: BEGIN 234 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide DECLARE KeyVar INT; SET KeyVar = EINOTEID; DELETE FROM MRL_EMP_EIVD WHERE EINOTEID = KeyVar; END P1 Running Virtual Agents Against Virtual Documents Changes made to Virtual Documents by running a Virtual Agent against selected documents in a view will produce updates in the documents immediately. However, if the Virtual Agent is updating values contained within a View column, that view will not immediately reflect the changes made to the documents. In order to see these changes, users will need to rebuild their index using the Shift + F9 function. If you are running a Virtual Agent against an open Virtual Document, changes can be viewed after the document has been closed and reopened. This is similar to the behavior expected when running a native Domino Agent against an open document. Chapter 13: Virtual Documents Activity 235 Chapter 14 Virtual Agents Activity There are three Advanced RealTime Activities in LEI. • Virtual Fields (previously known as LEI RealTime or DECS RealTime) • Virtual Documents • Virtual Agents This chapter provides information about the Virtual Agents Activity. Introduction to Virtual Agents Activities A Virtual Agents Activity creates Domino Agents that run stored procedures on the external system database. The Agents that it creates are added to the Domino Agent menu. If the stored procedures require parameters, you must create Domino documents that hold these parameters, then you select the document or documents and run the agents against them. Virtual Agents Activities execute external stored procedures in two ways: • Without parameters • Against selected documents that provide parameter values Procedures that do not accept parameters do not run against any documents, they are created and specified as agents that “Run on all documents in database.” Procedures that do accept parameters are created as agents that “Run on selected documents,” and require that at least one document be selected. All documents selected for virtual agents are used as a source for input parameters for the agent, and the input parameters for the external stored procedure are obtained from the correspondingly named fields in the target document. For example, if three documents are selected, the external stored procedure will be run three times, once for each document using input parameters obtained directly from the relevant document. For each document the Virtual Agents is run against, the items corresponding to the procedure parameters are retrieved from the document and provided as input parameters for the procedure. 237 Stored procedures in an external system are defined by a name and optional inputs and outputs. Input data is always provided as parameters, while output data can be returned either as output (or input/output) parameters or returned result sets. Agents in Domino do not directly support parameters, but the ability in Domino to run an agent against one or more documents provides the equivalent functionality. Note Use the output parameters of a Virtual Agent Activity to modify a non-virtualized Notes document or non-virtualized fields in a Notes document receiving output parameters. Note If the database is already open, the Virtual Agent will not be refreshed. Note After stopping the activity, the agents will no longer appear. However, if the database was open in the client, the Actions menu will continue to display the agents until the database is closed and reopen. Stored Procedure Parameters If the stored procedures that the agents run against require arguments, you must create Domino documents that have fields that are named exactly as the parameters in the external system stored procedure are named. Once these fields are set up and the appropriate values are stored in them (for input parameters), the agents can be run. Creating a Virtual Agents Activity Using the User Assistant You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Agents Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. This section describes how to create a Virtual Agents Activity using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” 238 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 1: Open the Activity Document In either the Activity View or the Favorites View (or a Folder View), click the Add Activity icon in the Action bar. A message box opens that explains the steps the User Assistant will guide you through. If you have not made any selections from the User Assistant, clicking the Cancel button on this message box closes the activity form and returns you to the LEI Administrator. Once you have made one or more selections while you are in the User Assistant, clicking Cancel causes the User Assistant to shut down, but the activity document stays open, in edit mode, displaying the choices you have made up to that point. You can then manually complete the activity document or abandon it. Note Any choices you make while using the User Assistant can be changed later by opening the activity document in edit mode. Chapter 14: Virtual Agents Activity 239 Step 2: Select the Domino Database to Monitor A list of Domino databases appears. Select the Domino database to monitor. Note When the list is large, you can scroll down the list to find the Domino application (nsf) you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Monitoring uses the Notes replication ID of the application database. A Virtual Agents activity does not distinguish between the databases that share the same replication ID. Databases have the same replication ID when they are replicas of each other and when one database is created using the operating system to copy an existing database. 240 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 3: Select the Lotus Connection for the External Data Source Select the Connection for external data source you want to monitor from those available in the Select Connection dialog box, shown below. Note When the list is large, you can scroll down the list to find the table you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Chapter 14: Virtual Agents Activity 241 Step 4: Select the Owner When prompted, select the metadata owner. Step 5: Name the Activity Document When prompted, enter a unique activity name and click OK. 242 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide After you name the activity and click OK, the following informational message box appears outlining your options. Click OK. The Virtual Agents Activity document appears, showing your selections. The numbered icons in the document correspond to the major steps the User Assistant uses to create the activity. Step 6: Select Options and Event Options Select the options and the event options you want in the “Options” and “Event Options” tabs in the document. See the section “Virtual Agents — Options” later in this chapter for more information. Step 7: Select Scheduling Options Select how you want to control the scheduling of the Virtual Agents activity. The options are Manual, AutoStart, and Custom. See the “Scheduling” section later in this chapter for more information. Step 8: Save and Close the Document Choose File - Save to save this Virtual Agents activity definition. Choose File - Close to close the Virtual Agents Activity document and return to the activity administrator. Chapter 14: Virtual Agents Activity 243 Step 9: Process the Virtual Agents Activity You can process the activity using the Start and Stop buttons in the Action bar of the Activities View or the Favorites (Folders) View. (When you select “Auto Start” in the Scheduling section of the activity document, the Virtual Agents activity is established each time that the server starts. See the Scheduling section for more information.) Click the Virtual Agents activity to select it and then click the Start button to begin processing it. To stop a Virtual Agents activity, select the Virtual Agents activity and click the Stop button. Creating a Virtual Agents Activity without the User Assistant You can optionally enable the User Assistant to prompt you the through the creation of a new Virtual Agents Activity. You can toggle the User Assistant on and off using the Help option in the Navigator panel of the Administrator. If you disable the User Assistant, a blank activity document is displayed, which you can fill in to define the activity. This section describes how to create a Virtual Agents Activity without using the User Assistant. For options not described in these steps, see the chapters entitled “LEI Administrator” and “Introduction to LEI Activities.” Step 1: Open the Activity Document Click the Add Activity icon. The Virtual Agents Activity Document appears. Step 2: Enter a Name for the Activity Enter a unique name for the activity in the Name field at the top of the document. Step 3: Select the Domino Application to Monitor Click the down-arrow icon in the “Domino Application” section. A list of databases appears. Select the database that you want to monitor. Note When the list is large, you can scroll down the list to find the Domino application (.nsf) you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” 244 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Note Immediate or “real-time” monitoring uses the Notes replication ID of the application database. A Virtual Agents activity does not distinguish between the databases that share the same replication ID. Databases have the same replication ID when they are replicas of each other and when one database is created using the operating system to copy an existing database. Step 4: Select the Lotus Connection for the Data Source Click the down-arrow icon in the External System section to select from existing Lotus Connection documents. Note When the list is large, you can scroll down the list to find the table you want to select. However, if the list is too large to be completely accommodated by scrolling, an overflow message appears at the bottom of the list (<overflow…>) and a box appears under the list, which you use to type in the exact name of the table you want to use. For more information on browsing and browsing requirements, see “Browsing” in the chapter entitled “LEI Administrator.” Step 5: Select Options and Event Options Select the options and the event options you want in the “Options” and “Event Options” tabs in the document. See the section “Virtual Agents — Options” later in this chapter for more information. Step 6: Select Scheduling Options Select how you want to control the scheduling of the Virtual Agents activity. The options are Manual, AutoStart, and Custom. See the “Scheduling” section later in this chapter for more information. Step 7: Save and Close the Document Choose File - Save to save this Virtual Agents activity definition. Choose File - Close to close the Virtual Agents activity document and return to the Virtual Agents activity administrator. Step 8: Process the Virtual Agents Activity You can process the activity using the Start and Stop buttons in the Action bar of the Activities View or the Favorites View. (When you select “Auto Start” in the Scheduling section of the activity document, the Virtual Agents activity is established each time that the server starts. See the Scheduling section for more information.) Click the Virtual Agents activity to select it and then click the Start button to begin processing it. Chapter 14: Virtual Agents Activity 245 To stop a Virtual Agents activity, select the Virtual Agents activity and click the Stop button. Virtual Agents Activity Document Options Once you have specified a Domino database and the connection you want to use, there are a number of options for you to consider in setting up a Virtual Agents activity. In this section, the options are organized into two major groups: • General Options • Integrated Credentials See the chapter entitled “Introduction to LEI Activities” for a description of the following sections that are common to all Activity documents: • Activity Name • Scheduling • Category • Comments General Options in Virtual Agents The General Options section of the Virtual Agents activity document, illustrated below, presents options that can be applied to a Virtual Agents activity. Each of the General Options is described after the illustration. Note Stored procedures in the external system must not have any parameters if they are to be used as Virtual Agents. 246 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide General Options Description External Owner If you specify an external owner, the choice of stored procedures Name is limited to those owned by this owner. Click the down arrow to browse the list of owners. Note Specifies the owner of the external system. You cannot browse the Agent Pattern List unless an External Owner Name is specified. Agent Pattern List LEI runs only the agents you select. Click the down arrow to open a dialog box where you can select from a list of agents or type in wild cards (use * or ? for pattern searches). Max. Connections This option sets the maximum number of connections to the external database that can be opened to service concurrent user requests simultaneously (that is, users creating, opening, updating, and deleting documents at the same time). LEI opens one connection to the external data source when the first Notes application event occurs. If two or more events occur simultaneously, additional connections are made, up to the maximum number of connections specified by this option. When the maximum number of connections is reached, subsequent events are queued and occur when each preceding event is serviced. Each connection lasts only as long as necessary to read from or write data to the external data source. While the connection is persistent, the time required to service each event is minimal and depends on the amount of data being read or written. The maximum number of connections, therefore, does not need to be very large in order to service multiple events. Lotus recommends that you set the maximum connections to 2 or 3, and if users experience significant delays, you can increase this number. Shutdown Options Delete Agents on Shutdown: This option instructs the activity to delete its virtual agents at shutdown. Leaving this option deselected takes advantage of noteid reuse. When the activity is restarted, a new Virtual Agent document will not be created if one already exists with the same name. Instead, the noteid is reused, but the contents of the agent document are recreated to catch modifications of the stored procedure in the external system. Output Parameters Save Output Parameters: This option instructs the activity to save the output parameters returned by a stored procedure to the selected documents. If the stored procedure also modifies data in the external system, care should be taken to ensure that it does not modify fields that are mapped to a selected virtualized record. In this situation, saving the output parameters to that record my overwrite the modifications made directly to the external system. Chapter 14: Virtual Agents Activity 247 Integrated Credentials in Virtual Agents User credentials for the external system are automatically looked up when the Integrated Credentials options is selected. The credentials are used to connect to the external system. Note The integrated credentials database is described in the Introduction to Activities chapter of this document. See the section entitled “Integrated Credentials Database” for information about setting up and using integrated credentials. Note Virtual Agents only use integrated credentials for agent execution. Each of the Integrated Credentials Options is described in the table following the illustration. 248 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Integrated Credentials Options Description Use Lotus Connector for xyz Credentials Uses the user ID and password that is specified in the Connection Document you are using in this activity. Lookup Credentials Missing Credentials — Enables you to specify how missing credential data should be processed. Use Connector Credentials — If you select this option and there is a credentials lookup failure, default credentials in the activity can be used. If you do not select this option, the operation can fail (this is the default behavior). Credentials DB Filepath — Enter the name of the credentials database to be used for storing Notes user names and corresponding external systems credentials. The path is specified relative to the LEI directory (for example leidir). Note You must have created this database using the template leicred.ntf. See "Setting Up and Using Integrated Credentials" in the Introduction to Activities chapter of this document. Executing Virtual Agents from HTTP Client There are two ways to execute a Virtual Agent if you are using a HTTP client; directly through a URL or by adding the agent to a Web object in the form as you design it. Using a URL You can execute it directly through a URL. For example, if your agent was called “Update_VA” and was on the server MyServer and used the names.nsf database, the URL command would be: http://myserver.mysite.com/names.nsf/Update_VA?OpenAgent Using a Web Object in the Form Design The other way to use your virtual agent via HTTP client is to enter your agent within the WebQuerySave or WebQueryOpen object in Designing your Form. For example: @Command([ToolsRunMacro]; "<Your agent goes here>") Each time a form is saved or opened from the Web your Virtual Agent will launch. Chapter 14: Virtual Agents Activity 249 Running Virtual Agents Against Virtual Documents Changes made to Virtual Documents by running a Virtual Agent against selected documents in a view will produce updates in the documents immediately. However, if the Virtual Agent is updating values contained within a View column, that view will not immediately reflect the changes made to the documents. In order to see these changes, users will need to rebuild their index using the Shift + F9 function. If you are running a Virtual Agent against an open Virtual Document, changes can be viewed after the document has been closed and reopened. This is similar to the behavior expected when running a native Domino Agent against an open document. Running Virtual Agents Against Virtual Fields Changes made to Virtual Fields documents by running a Virtual Agent will produce updates in the specified documents immediately. However, if the Virtual Agent is updating values contained within a View column, that view will not reflect the changes made to the documents, and pressing Shift + F9 will not refresh the view. In order for Virtual Fields to be displayed within a view, you must have selected “Leave all/selected Real Time fields in document” when building the activity. This, in effect, makes native copies of the fields within the document. These fields will not be updated or reflect changes made in the external system until the Notes document is updated manually or by using a formula or script. 250 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 15 Replication Activity This chapter provides information about the LEI Replication Activity. Introduction to the Replication Activity A Replication Activity synchronizes databases by updating one database from the data in another. There are two types of replication available within the Replication Activity form: • Primary Key Replication • Timestamp Replication A condition can also be used in the Replication Activity document’s Conditional Clause field to further refine the result set to achieve an enhanced, selective replication. See the section “Types of Replication Activities” later in this chapter for more information. You can control how the Activity resolves conflicts, which in LEI terms are simply instances where a data item in one database does not exactly match a data item in another database. When to Use the Replication Activity The Replication Activity is recommended for use when you want to do either of the following tasks. • Synchronize data in two databases • Merge data from one data source into another data source You can also use the Replication Activity in conjunction with a Polling Activity to refine when the Replication Activity occurs. For example, you could poll a data source for a specific event, such as a data insertion, and on detection of that event start the Replication Activity that replicates the new data in another data source. 251 Types of Replication Activities There are two main types of replication activities — primary key and timestamp. Primary Key Replication LEI primary key replication replicates data based on a unique key common to both connections that can be composed of one or more fields in the metadata. The function of the primary key is to determine if matching records exist in both data sources and if an update to a record, the insertion of a new record, or the deletion of a record is required. When replication occurs, records are replicated according to the following default rules: • If a record with matching key values exists in both connections, and the data field values are identical, no action is taken. • If a record with matching key values exists in both connections, and the data field values are different, the Target record is updated. • If a Source record contains key values that are not present in the Target, a new record is inserted into the Target. • If a Target record contains key values that are not present in the Source, that record is deleted from the Target. Note In primary key replication, the Replication Activity never changes records in the Source connection. Timestamp Replication You can control replication based on timestamp fields provided in the Activity Document using the Enable Timestamp Replication option. Using this option requires that the two connections specified on the Activity Document (Source and Target) contain at least one timestamp field. The Replication Activity Document maintains internal timestamp fields based on the last replication from each respective connection. These fields are SrcTimeStamp and DestTimeStamp. You can see these fields by viewing the document properties of the Activity Document. To do so, right-click 252 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide while active in the Activity Document or choose File - Document properties from the Notes top bar. Use the Fields tab and scroll down to locate these field names. In timestamp replication, there is no designation of “source” and “target”. Replication actions can affect both connections. Because the connections are designated on the Activity Document as Source and Target, we use these terms when explaining the following rules. When replication occurs, records are replicated according to the following default rules: • If a Source record contains a timestamp value that is later than the SrcTimeStamp value, and a Target record does not exist with the same key fields, a new record is inserted into the Target. • If a Target record contains a timestamp value that is later than the DestTimeStamp value, and a Source record does not exist with the same key fields, a new record is inserted into the Source. • If a Source record contains a timestamp value that is later than the SrcTimeStamp value, and a Target record exists with the same key fields, the Target record is updated. • If a Target record contains a timestamp value that is later than the DestTimeStamp value, and a Source record exists with the same key fields, the Source record is updated. • Records are never deleted from either connection. Note If a Source and Target record share the same key fields and contain timestamp values that are later than the SrcTimeStamp and the DestTimeStamp respectively, then the Target record is updated. In this case, the Replication Activity does not use the timestamp values to determine the replication action. Chapter 15: Replication Activity 253 Understanding How Timestamp Replication Works The first time that a timestamp replication occurs, all records in both connections are replicated according to the above rules because the initial values of SrcTimeStamp and DestTimeStamp are 01/01/0001 12:00:00 AM. Subsequent replications adhere to the rules already described with the following consideration. Replication action is determined using a timestamp window; not strictly based on whether the record to be replicated has a timestamp value that is greater than SrcTimeStamp or DestTimeStamp. In order for a source or target record to be replicated, its timestamp value must be greater than the respective internal timestamp value (SrcTimeStamp or DestTimeStamp) but must also be less than the current system time of the database server. Consider the following example: In the Replication Activity Document, the Source connection is DB2 and its designated timestamp field is LASTUPDATED. The Source connection contains a record where LASTUPDATED is currently 03/27/02 11:31:46 AM. The SrcTimeStamp field value is 03/27/02 10:47:07 AM. The current system time of the DB2 server is 03/27/02 11:29:12 AM. This record will not replicate. Although the 03/27/02 11:31:46 AM is greater than SrcTimeStamp (03/27/02 10:47:07 AM) it is not less than the current system time of the DB2 server (03/27/02 11:29:12 AM). You can avoid this situation by ensuring that all your timestamps (especially those set with a trigger or formula) are in synch with one another. Note The DB2 column used as the TIMESTAMP in timestamp replication must be of type TIMESTAMP, not DATE. Using Timestamp Replication with Notes Databases During timestamp replication, the Replication Activity uses the Domino server time where the Notes database resides as the current system time to determine if records should be replicated. It therefore becomes important to use a timestamp field that captures this time. Using @Modified retrieves the modified date/time for that document which was set by its Domino server when the document was saved. @Modified comes from the server. @Now comes from the desktop. Because the Domino server time and the Notes client workstation may not be in exact agreement about what time it is, it is probable that you will miss some documents during timestamp replication if the time on the client is earlier than the time on the server. Therefore you must use @Modified to obtain the proper time in any field formulas which will be used for timestamps. However @Modified is always one edit behind because the modified timestamp is the last thing to be set when the document is saved. @Modified reads the modification time, but because the document contains the last 254 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide time the document was saved at the time the formula is executed, it will not return the time from the current save. There are two options for solving this problem, as shown below: • The first is to use the Notes Connection Option “Load last modified TimeStamp (@Modified).” In this case, @Modified would be entered into the timestamp field in the Replication Activity Form. This method requires no special changes to the Notes application. This option is discussed in more detail in the chapter on Notes connections. • The second option involves modifying your Notes form to add a QueryClose event. In your timestamp field (which should be a computed field), use the following formula (where “EditedDate” is the fieldname of the field used for timestamp): @IF(@IsDocBeingSaved;@Modified; EditedDate) This formula places the modification timestamp from the Domino server into the field “EditedDate” if the document is saved, otherwise it doesn’t change. To have the field contain the real modified date, not the one from the previous edit, create a QueryClose event script for the form. This event will perform two saves of the document. The first will set the EditedDate field to the modified time of the last edit. The second save will set the modified time to the time of the save just performed. It is possible to get to within a tenth of a second of the real time. Sub Queryclose(Source As Notesuidocument, Continue As Variant) Dim workspace As New NotesUIWorkspace Dim doc As NotesUIDocument Set doc = workspace.currentdocument If doc.editmode Then Call doc.save Call doc.save Else Stop End If End Sub Chapter 15: Replication Activity 255 Using Database Triggers with Timestamp Replication Database applications often use triggers to set timestamps when a row of data has been modified. When LEI replication updates a row, the trigger may reset the timestamp to the time modified in that file or database. This may cause unnecessary replications of this row back to the other database the next time replication is run. The trigger should be designed to detect whether the client application or LEI is updating the row and should handle the timestamp accordingly. Specifying a Selection View in a Notes Database Timestamp replication creates a temporary view based on the selection criteria of documents to be replicated. The Notes Connection Options field “View to Use for Selection” lets you specify a permanent view to be used to select the documents that the Activity will use. Supplying a permanent view name can dramatically improve performance by not requiring the database to be re-indexed every time the Activity runs. LEI requires the ability to modify this view, so it should not be a view that users also access. Sort Order Considerations Replication produces result sets from both connections and scans the result set a single time for corresponding keys. If the connections use different sort orders, then replication may pass over keys that are out of order thus producing unnecessary inserts or deletes. To avoid this, the Order Metaconnector is available to impose a consistent sort order on result sets. The Order Metaconnector should be used when the following conditions are met, in which case it must be used for both connections: • One or more keys are text. Sort order is not a problem for non-text keys. • The connections in the replication sort differently. This varies by Connector. Some may use the same sort order even though they connect to different systems. • The text keys contain characters that are included in those which sort differently. For example, Lotus Notes and Oracle may sort certain punctuation characters differently, but if the text keys are purely alphanumeric, the sort order differences may be irrelevant for this replication. Note If the data being replicated is not on the same system and both connections use the same Connector (for example, DB2 to DB2 replication), the Order Metaconnector is not needed. Note When data is obtained from an EBCDIC-based system, the sort order will differ from a non-EBCDIC system. 256 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide You should ensure that the sort orders specified for the databases used in the replication are identical. One option for handling sort order differences is to use a numeric key. You can also use the Order Metaconnector to gain further sort order control. For issues arising from sorting differences based on case-insensitive string comparison, see the section entitled, “Field Mapping Options” later in this chapter. How to Create a Replication Activity The Replication Activity Document defines information needed for a Replication Activity. This information includes replication parameters and options for controlling data treatment. The steps below outline the general procedure for creating a Replication Activity. See the other sections in this chapter for more information about the particular fields and options on the Replication Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Replication from the resultant list. 3. Fill in the required fields and select the desired options for the Activity in the Replication Activity Document. Note All connections referenced in the Activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. There is a complete example of building a Replication Activity in the “Tutorials” appendix. Chapter 15: Replication Activity 257 Replication Activity Document The Replication Activity Document is shown below. To create a new Replication Activity, click Add Activity and then select Replication from the resultant list. To open an existing Replication Activity, double-click its name in an Activities view. Note See the chapter entitled “Introduction to LEI Activities” for descriptions of the Activity Execution Options, Scheduling, and Other options. 258 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Special Settings The Replication Activity has the following optional settings in the Activity Document action bar. Button Description Trial-Run Replication Default Settings Clicking this button performs a standard replication; it clears the fields associated with Trial-Run Replication, effectively returning some sections of the Replication Activity document to their default settings. This deselects the Skip Conflicts option, deselects all Connection Restrictions, and sets the Source Connection as master. To return to non-trial replications, select the Default Settings option and click OK when prompted. The Default Settings option clears the settings associated with Trial-Run Replication and restores the Activity to its default settings. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. The Source and Target sections provide information on the data source and destination. Connections must be created before they can be used in this Activity Document. Source The Source Connection fields are described below. Field Description Connection Specifies the source database, by defined connection name, from which data will be selected and copied. The connection identifies the server and database with access information. The Additional Information segment of the document (see below) identifies the metadata. To display a list of available connections, click on the arrow to the right of the Source heading. Edit Connection Opens the selected Connection Document in Edit mode. Metadata Type Specifies the metadata, based on the type of connection chosen. For example, in a relational database the metadata is a Table; in a Notes database the metadata is a Form. continued Chapter 15: Replication Activity 259 Field Description Source Restrictions Provides options that you can choose from to restrict the operational behavior of the Replication Activity relative to each connection. For an extended example of how to work with restriction settings, see “Example Results for Restricted Settings” later in this chapter. w Skip Insertions — Prevents any records in the Source connection from receiving inserts during replication. w Skip Updates — Prevents any records in the Source connection from being updated during replication. w Skip Deletions — Prevents any records in the Source connection from being deleted during replication. Enabling Skip Insertions, Skip Updates, and Skip Deletions causes replication for that connection to not allow writeback modifications. This reduces the locking level, and may be used to avoid locking one side of a replication when it is known that no changes will be made to that side. This is especially useful when replicating from non-updateable views in a relational database. Note These options are only available if you have selected Timestamp Replication Target The Target Connection fields are described below. Field Description Connection Specifies the destination database, by defined connection name, to which data will be copied. To display a list of available connections, click on the arrow to the right of the Target heading. Edit Connection Opens the selected Connection Document in Edit mode. This button is available only if you are in Edit mode. Metadata Type Specifies the metadata, based on the type of connection chosen. For example, in a relational database the metadata is a Table; in a Notes database the metadata is a Form. continued 260 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Target Restrictions Provides options that restrict the operational behavior of the Replication Activity relative to each connection. For an example of how to work with restriction settings, see “Example Results for Restricted Settings” later in this chapter. w Skip Insertions — Prevents any records in the Target connection from receiving inserts during replication. w Skip Updates — Prevents any records in the Target connection from being updated during replication. w Skip Deletions — Prevents any records in the Target connection from being deleted during replication. Enabling Skip Insertions, Skip Updates, and Skip Deletions prevents replication, for that connection, from allowing writeback modifications. This reduces the locking level and may be used to avoid locking one side of a replication when it is known that no changes will be made to that side. This is useful when replicating from non-updateable views in a relational database. Mapping The Mapping section of the Activity Document enables you to specify either manual or automatic field mapping. The default is manual mapping. Both mapping styles are described here. For related information, see the Data Mapping section in the chapter entitled “Introduction to LEI Activities.” Manual Mapping The manual mapping fields are described below. Chapter 15: Replication Activity 261 Field Description Source Key(s) Specifies the fields or columns of the data that together represent a unique primary key for that data collection. No two records can have the same value for all key fields, and at least one key field is required. The key fields for both connections must correspond. The Source Key and Target Key value must match. If using a Notes Connection as Source, see the “Metadata Creation: Select Target Metadata” description for the Data Options tab. Target Key(s) Specifies a corresponding key field as that used for the Source Key(s) value. Source Column/Fields(s) Specifies the columns or fields used in the Source connection. Target Column/Field(s) Specifies the columns or fields used in the Target connection. Automatic Mapping When you select the Automatic option, the automatic mapping fields appear. They are described below. Field Description Source Key(s) Specifies the fields or columns of the data that together represent a unique primary key for that data collection. No two records can have the same value for all key fields, and at least one key field is required. The key fields for both connections must correspond. The Source Key and Target Key value must match. Target Key(s) Specifies the corresponding key field as that used for the Source Key(s) value. By Name Maps data from each source column to an identically named column in the destination metadata. To use this option, field names in the Source and Target connections must be identical or errors will occur. By Position Maps data from each source column to the destination by column sequence: first column to first, second to second, and so on. To use this option, the number of field names in the Source and Target connections must be identical or errors will occur. Positions must be identical; left to right or top to bottom. 262 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Replication Options There are several options available to you in the Replication Options section of the Replication Activity Document. Each subsection is described below. Timestamp Options If you select the Timestamp tab and then the Enable Timestamp Replication option, the following choices are available. As well, the three Source Restrictions choices (Skip Insertions, Skip Updates, Skip Deletions) become available. Chapter 15: Replication Activity 263 Field Required Description Timestamp Replication: Enable Timestamp Replication No Selecting this option enables timestamp replication. Primary key replication is the default. Reset Replication Timestamp No Selecting this option reinitializes the replication timestamp of both metadata timestamps (SrcTimeStamp and DestTimeStamp) to 01/01/0001 12:00:00 AM. This forces the next timestamp replication to behave as though no previous replication has occurred and all records will be replicated. The option is useful if replication has fallen out of sync. For more information, see the rules at the beginning of this chapter. Source Timestamp Required for This specifies the name of the field that Timestamp holds the record’s timestamp value. If a Replication field is entered here, the activity compares both the timestamp and the primary key. If used, a timestamp field name must be entered for both the Source and the Target. Source: Replicate Timestamp Field Optional If you enable the Do Not Replicate option, only the data field values, and not the timestamp field value, are replicated. This prevents replication of records from occurring when the timestamp field value changes but the data field values do not. The activity will not alter the timestamp field; any change is left to the data source to manage. Target Timestamp See Source Timestamp definition. Target: Replicate Timestamp Field See “Source: Replicate Timestamp Field”. 264 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Logging Options When you select the Logging Options tab, the following options appear, enabling you to log errors that occur during replication. This is helpful when troubleshooting. Field Description Log Key Values on Error If enabled, this creates an entry in the activity log when an error occurs during replication. The entry contains the keys from the record that generated the error. Conflict Options The Replication Activity Document provides options for specifying how to resolve conflicts that occur while the Activity runs. The term conflict specifically refers to a data item in one database not exactly matching a data item in another database. A conflict occurs when the timestamp cannot resolve a “winner.” In keyed replication, all differences are categorized as conflicts. In a timestamped Activity, only certain discrepancies are categorized as conflicts. Conflicts can be a normal event during replication. A decision must then be made as to how to resolve this conflict. In primary key replication, the Source connection is the one that wins any replication conflict. Note You can optionally choose to log replication conflicts to a log file. These logged entries are not errors, but are rather instances of a conflict event. Listing name conflicts to a log file helps you to both check the Activity and resolve any naming mismatches between the two databases. LEI allows you to control how replication conflict events are handled. When you select the Conflict Options tab, the following options appear. Chapter 15: Replication Activity 265 Field Description Conflict Handling Saves a document containing the conflict in the non-master metadata. When you select the Save Conflict Data to Metadata, the form displays the Conflict Link: Metadata Name choices described below. Conflict Link: Metadata Name Specifies that the conflict keys and data be saved to a specific table or form (metadata) in either the Source or Target connection. You must specify a table or form name using the browse and select button to the right of the “Metadata Name” field name. The metadata name that you specify receives the conflict loser records. w Source — specifies that the metadata name is in the Source Connection w Target — specifies that the metadata name is in the Target Connection Log Conflicts Logs conflicts in the Activity log. When you select Enable Conflict Logging, the form displays the Logging Options and Limit To choices described below. Logging Options Specifies how conflict logging will be performed. Limit To w As Event — logs conflicts as errors in the Activity Log w As Error and Continue — logs conflicts as errors in the Activity Log, but continues the Activity w As Error and Stop — logs conflicts as errors in the Activity Log and stops the Activity Using the As Error options is slower, since each log entry is written to disk. Using the As Event options is much faster and is recommended for production environments. Used to limit logged conflicts to a useful number for the given Activity. When this number of conflicts has been logged, additional conflicts are not added to the Activity log. The default is 500. Specifying a value of zero indicates no limit (log all conflicts). 266 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Data Options When you select the Data Options tab, the following options appear. Field Description Metadata Creation Create Target Metadata When you select “Create Target Metadata”, the “Index to Create on Metadata Creation” field appears. Note When using a Notes connection as the Source Connection and this option, the activity creates Target text fields of 64996 bytes, the maximum size of a Notes Text field. This causes legitimate errors with some RDBMS external systems during replication. To avoid these errors, do not use a text field as a Source Key Field. Index to Create on Metadata Creation When the option to create metadata is selected and metadata creation is performed, an index will also be created on the new metadata. The name entered is the name for the new index. The Replication Activity key fields are the index columns. Index creation may not be supported by all Lotus Connectors. An index is crucial for good performance against some connections, such as relational databases. continued Chapter 15: Replication Activity 267 Field Description Data Comparison and Trimming Specifies how data is to be compared and how trailing spaces are to be processed. Reduced-precision Comparison — Avoids false mismatches as a result of differences between database types in the precision used for storing datetime or timestamp and floating point values. One database, for example, may recognize fractions of seconds and another not. This option uses a lower common degree of precision. Datetimes are compared to the second and without time zone or daylight savings time. This performs date/time comparisons based on year, month, day, hour, minute, and second only. It also performs floating-point comparisons to ten digits of precision. This option does not perform comparisons based on time zones, daylight savings time, or hundredths of a second. This option is useful for replication between databases that do not support the same degree of date/time floating point precision. Case-insensitive String Comparison — This option is intended for use only when both connections in a replication use case-insensitive sorting (such as LEI replication between two Notes connections). In this situation, normal replication case-sensitive comparisons can cause false conflicts. Note Using this option causes replication to compare text without case sensitivity. Note This option does not cause the connections to perform case-insensitive sorting. Disable Trimming of Text Trailing Spaces — Specifies that text fields that contain trailing spaces will not be trimmed. continued 268 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Field Description Conditional Clause for Source Refines the Activity by including an optional condition clause in the statement that is executed against the result set based on key field(s). For example, a SQL WHERE clause or a Notes selection formula clause can be used. You can enter a condition for either or both connections. The syntax of a conditional clause is defined by the connection, however, it is not necessary to include any keywords (such as SELECT or WHERE). The conditional clauses act as filters for determining which records are affected in both the Source and Target. The clauses limit the records in the Source and Target that are compared during a Replication Activity. As an example, if you have a Source and a Target table, each with 500 employee records with EmpIDs from 1-500, the following is true: If your Conditional Clause for Source is set to EmpID > 250, the Replication Activity will only use, for comparison, records in the Source with EmpIDs greater than 250. Updates, inserts and deletes will then be performed accordingly on the Target based on a comparison between records with EmpIDs greater than 250 in the Source and the entire Target. Conditional Clause for Target See the description for “Conditional Clause for Source” above and the following. As an example, if you have a Source and a Target table, each with 500 employee records with EmpID’s from 1-500, the following is true: If your Conditional Clause for Target is set to EmpID > 250, the Replication Activity will only use, for comparison, records in the Target with EmpIDs greater than 250. Updates, inserts and deletes will then be performed accordingly on the Target based on a comparison between the entire Source and records with EmpIDs greater than 250 in the Target. Chapter 15: Replication Activity 269 Using the Do Not Replicate Timestamp Option The following table describes possible results when using the Replicate Timestamp Field: Do Not Replicate settings found on the Timestamp tab on the Replication Activity Document. Source Connection Do Not Replicate Setting Target Connection Do Not Replicate Setting Action on Record Result of Replication on Record Enabled N/A Update Source timestamp field value only No action Enabled N/A Update Source timestamp field value and data field values Update data field values only in Target Not Enabled N/A Update Source timestamp field value only Update timestamp field value only in Target Not Enabled N/A Update Source timestamp field value and data field values Update data field values and timestamp field value in Target N/A Not Enabled Update Target timestamp field value only Update timestamp field value only in Source N/A Not Enabled Update Target timestamp field value and data field values Update data field values and timestamp field value in Source N/A Enabled Update Target timestamp field value only No action N/A Enabled Update Target timestamp field value and data field values Update data field values only in Source 270 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide If the connection is using a timestamp that is set by the external system database through a trigger or as a result of saving the document, the Activity has no effect on that timestamp. It is set per the rules of the external system database. As an example, when using the “Modified” time on a Notes document, selecting the “Load Last Modified Timestamp (@Modified)” Connection Option will force the Activity to use the Modification time on the Notes document. If the document is changed during a replication, Notes will update that modification time as the document is saved. When a user or data entry application outside of LEI enters new data or changes record data in a database, the timestamp field value is updated to reflect the access date and time at which the latest modification occurred. You can control whether or not the Activity updates the timestamp field in the Target connection by using the “Do Not Replicate” option on the Source connection. This example table shows results when both connections are Notes and are using the “Load Last Modified Timestamp (@Modified)” Connector Option. Source Connection Do Not Replicate Setting Target Connection Do Not Replicate Setting Action on Record Result of Replication on Record Enabled N/A Change Source timestamp field value only Does not update timestamp field value in Target Enabled N/A Change Source timestamp field value and data field values Update data field values only in Target, timestamp field value set to time of document change (Target is now later than Source by Notes when document was modified). N/A Not Enabled Change Target timestamp field value only Update timestamp field value only in Source N/A Not Enabled Change Target timestamp field value and data field values Update data field values and timestamp field value in Source Not Enabled N/A Change Source timestamp field value only Update timestamp field value in Target (Target is now later than Source) continued Chapter 15: Replication Activity 271 Source Connection Do Not Replicate Setting Target Connection Do Not Replicate Setting Action on Record Result of Replication on Record Not Enabled N/A Change Source timestamp field value and data field values Update timestamp field value and data field values in Target N/A Enabled Change Target timestamp field value only Does not update Source N/A Not Enabled Change Target timestamp field value and data field values Update data field values and timestamp field value in Source N/A Not Enabled Change Target data field values and timestamp field value Update data field values and timestamp field value in Source Not Enabled N/A Change Source data field values and timestamp field value Update data field values and timestamp field value in Target N/A Enabled Change Target timestamp field value only Does not update Source Enabled N/A Change Source timestamp field value only Does not update Target 272 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Using Restrictions in Keyed Replication Activities Settings for replication restrictions apply to the connection for which they are set. For example, when options are set on the Target connection, changes to the Source will not affect the Target for the specified action. When the Trial Run Replication option is used, all replication restrictions are ignored. In keyed replication, the Target connection can benefit from restriction settings, but not the Source. The following tables represent the effect of setting various replication restrictions on the target connection. The action performed when each option is set will produce the specified results to the data in the connection for which the option is set. The type of replication also determines behavior of the destination. Keyed replications behave differently than timestamp replications. All of these options are used to manipulate the data transfer in a way that will cause the data to be different on each side of the replication. Target Connection Settings Action Result Skip Insertions Inserts into Source Does not insert into Target Inserts into Target Deletes from Target Updates Source Does not update Target Updates Target Does not update Target Deletes from Source Retains in Target Deletes from Target Replaces with Source Skip Updates Skip Deletions Chapter 15: Replication Activity 273 Using Restrictions in Timestamp Replication Since timestamp replication is bi-directional (Source updates Target and Target updates Source), it is possible to set restrictions for either connection. “Skip Deletions” is required on both Source and Target connections for all timestamp replications, since deletions are never propagated to the other connection. It is assumed in this chart that Skip Deletions is set and all deletions are ignored by each connection. This table shows the Source Connection Settings. Source Connection Settings Action Result Skip Insertions and Deletions Inserts into Source Inserts into Target Updates Source Updates Target Inserts into Target Does not insert into Source Updates Target Updates Source Inserts into Source Inserts into Target Updates Source Updates Target Inserts into Target Inserts into Source Updates Target Does not update Source Inserts into Source Inserts into Target Updates Source Updates Target Inserts into Target Does not insert into Source Updates Target Does not update Source Skip Updates and Deletions Skip Inserts, Updates and Deletions 274 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide This table shows the Target Connection Settings. Target Connection Settings Action Result Skip Insertions and Deletions Inserts into Source Does not insert into Target Updates Source Updates Target Inserts in Target Inserts in Source Updates Target Updates Source Inserts into Source Inserts into Target Updates Source Does not update Target Inserts into Target Inserts into Source Updates Target Updates Source Inserts into Source Does not insert into Target Updates Source Does not update Target Inserts into Target Inserts into Source Updates Target Updates Source Skip Updates and Deletions Skip Inserts, Updates and Deletions Chapter 15: Replication Activity 275 Replication Examples This section provides some examples that employ the Replication Activity with replication restrictions. Skip Insertions In this example, “Skip Insertions” is set on the Target and a record is inserted into the Source. The insertion is ignored by the Target in the next replication. Use this option to prevent new records from being inserted into the Target. 276 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Skip Updates In this example, “Skip Updates” is set. A record is updated in the Source but is not replicated into the Target. Use this option to retain original data. Chapter 15: Replication Activity 277 Skip Deletions The example below shows the effect of “Skip Deletions.” The Source has deleted a record but the deletion is skipped in the Target. Use this option to preserve data in the Target that has been deleted from the Source. 278 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Chapter 16 Java Activity This chapter provides information about the LEI Java Activity. Introduction to the Java Activity A Java Activity allows LEI to launch a Java application. When to Use the Java Activity Use the Java Activity when you want to use LEI to launch a Java application. The Java application need not be restricted to LEI-specific applications. A programmer can choose any IDE to develop and deploy applications. When developing a Java Activity or Activity Document, consider the following: • The application must exist on the same machine as the LEI server that is designated to run that application. • A Java JVM must also exist on that machine. A Java Activity is useful in the following situations: • When you need more elaborate control of source and destination during data transfers • When you expect to perform different kinds of data massaging based on certain application-specific conditions Note The CLASSPATH variable on the machine that is running the Java Activity must point to the class referenced in the activity. The system path must reference the location of the JVM used for the activity. How to Create a Java Activity The steps below outline the general procedure for creating a Java Activity. See the other sections in this chapter for more information about the particular fields and options on the Java Activity Document. 279 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Java from the resulting list. 3. Enter the required class field and select the desired options for the activity in the Java Activity Document. 4. Save the Activity Document. Java Activity Document The Java Activity Document specifies the information required to run an LEI Java application. The Java Activity Document is shown below. To create a new Java Activity, click Add Activity and then select Java from the resultant list. To open an existing Java Activity, simply double-click its name in the Activities view. 280 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Note See the chapter entitled “Introduction to LEI Activities” for descriptions of Activity Execution, Scheduling, and Other options. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Java Use this section of the Java Activity document to identify the main Java Class and optionally the CLASSPATH and Virtual Machine that the Activity will use. Field Description Class Name Specifies the name of the Java Class to execute using any command line parameters. The class specified in the Class Name field should be in the CLASSPATH environment variable. Class Path (optional) Optionally prepends the existing CLASSPATH environment variable set on the server that will execute the Java application. Java VM (optional) Specifies an optional Java Virtual Machine (JVM) override. If left blank, java.exe will execute. You may enter the path to an alternate JVM to execute the Java application. Doing so may require a change in the CLASSPATH. Chapter 16: Java Activity 281 Chapter 17 Scripted Activity This chapter provides information about the LEI Scripted Activity. Introduction to the Scripted Activity A Scripted Activity is one that executes LotusScript (LSX) commands. LEI extends the LSX classes with additional methods to support the LEI Administrator and log databases. Using LotusScript classes allows you extend the functionality available from the other LEI Administrator Activity operations such as Direct Transfer, Polling, and Replication. You can use the LSX and a Scripted Activity to construct customized routines in situations where you need more complete control over source and destination data transfers. You can also use a Scripted Activity to perform data manipulation or evaluation during transfer. Follow these steps to create a Scripted Activity. Note The “Tutorials” appendix contains two excellent examples of creating a Scripted Activity. The first example illustrates how to use the two agents supplied in the LEI Script Vault (leivlt6.nsf) database in conjunction with the supplied leipackagetrack.nsf sample database. 1. Install the Notes client or Domino Designer client to access the LC LSX classes. 2. From the Lotus Notes or Domino Designer database Design Agents editor window (also known as the Integrated Development Environment or IDE) type the following in the options section of the script window. Uselsx "*lsxlc" 3. Within the IDE, create the LotusScript routine using LC LSX classes. Note Once you save the script and run it once, the system loads the LSX and the descriptions. The LC LSX classes appear automatically in the Domino Classes section of the script browser drop down list. 283 4. In the Agent Properties box, set the Runtime target to None (@commands may be used in this type of agent). 5. Within the LEI Administrator — Scripted Activity, identify the Lotus Notes database and server where the LotusScript routine has been stored. Proceed to schedule and execute the LEI Scripted Activity. See the Lotus Connectors LotusScript Extensions Guide for more information about the LSX. Script development requires that either a Notes client, Designer Client, and/or the LEI server be installed. In the Lotus Notes or Designer client, you can edit, compile, and test LC LSX scripts, with or without named sessions which requires LEI functions. However, only LC LSX scripts without named sessions can be deployed on the Notes client, since LEI functions are designed to be used for batch jobs. Scripts with named sessions are deployed as Scripted Activities. With LEI installed, scripts must read their connection information from the LEI Administrator. Use of the LEI Administrator and logging is established when the LCSession class object is created. Scripted Activities require that the first Lotus Connector class object created be an LCSession object. The LCSession object must be created with a name. Note With Scripted Activities, the activity name is used in the log. The session name is only used in the log when the Activity is run manually (not as part of a Scripted Activity). Use of the LEI Administrator changes the behavior of the LCConnection class. LEI requires that the name given when creating each LCConnection object be the name of an existing Connection Document. The connection parameters are loaded from the corresponding Connection Document. By contrast, the LSX requires that the name given when creating each LCConnection object be the Connector’s library name and all parameters must be assigned explicitly. When a scripted agent is called, it is calling a defined activity. This activity must have named Connection Documents associated with it. As a result, when the LCConnection object is instantiated with the following call, the <libraryname> takes on a new meaning within this context. It becomes the name of a connection to be used by the script. Dim <connectionname> as New LCConnection ( <libraryname> ) 284 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide How LEI Supports LotusScript The LEI LotusScript is first created as an agent using the Notes Integrated Development Environment (IDE), and later LEI executes the agent. The LC LSX extends the LotusScript language to support the complete LEI programming interface. Script development requires that either a Notes client, Designer client and/or the LEI server be installed. Note In the Lotus Notes or Designer client, you can edit, compile, and test LC LSX scripts, with or without named sessions which requires LEI functions. However, only LC LSX scripts without named sessions can be deployed on the Notes client, since LEI functions are designed to be used for batch jobs. Scripts with named sessions are deployed as Scripted Activities. When editing a LotusScript agent, the Notes IDE loads the LC LSX. This permits the LotusScript syntax to be verified. When the agent is executed or debugged, the LSX establishes a connection with the local LEI server or client to process database connections and perform logging. To use the LEI extensions to LotusScript, include the LC LSX within a scripted agent. Both the Notes and LEI program directories must be listed in the system path. Notes requires the LEI program directory to be in the path so the IDE can load the LSX for authoring and testing of the agent. LEI requires the Notes program directory to be in the system path so that the agent may be executed as part of the Scripted Activity. See Appendix E of the Lotus Connectors LotusScript Extensions Guide for details on LEI-specific behaviors, including descriptions of the LCSession and LCConnection classes. See the previous section entitled “How to Create a Scripted Activity.” Also see the examples in the “Tutorials” appendix of this manual. Running Unrestricted LotusScript Agents To execute unrestricted LotusScript agents you must grant permission to “*“. In addition, either “Default” or (more secure) the signer of the agent must have Editor access to the LEI Administrator database with permission to delete documents. Chapter 17: Scripted Activity 285 How to Create a Scripted Activity The steps below outline the general procedure for creating a Scripted Activity. See the other sections in this chapter for more information about the particular fields and options on the Scripted Activity Document. 1. Open the LEI Administrator database, decsadm.nsf. 2. Select Add Activity from the Administrator. Select Scripted from the resulting list. 3. Fill in the required fields and select the desired options for the Activity in the Scripted Activity Document. Note All connections referenced in the activity should exist and be valid. To test the validity of a connection, use CONTEST as described in the Lotus Connectors and Connectivity Guide. 4. Save the Activity Document. There are two examples of building a Scripted Activity in the Tutorials appendix. Scripted Activity Document The Scripted Activity document includes the name of the Notes agent that contains the LSX script to be executed, as well as the standard scheduling options common to all Activities. To view the agent, select the Notes database which contains the agent (the default is the LEI Script Vault). Next, choose View - Design from the top menu and then select Agents in the Navigator. 286 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide The Scripted Activity Document is shown below. To create a new Scripted Activity, click Add Activity and then select Scripted from the resultant list. To open an existing Scripted Activity, simply double-click its name in the Activities view. Identification Use the Name field to specify the unique Activity name. This name appears in the Administrator’s list of defined Activities. It also appears in the LEI server console’s list of defined Activities. Chapter 17: Scripted Activity 287 Script This section identifies the LotusScript that the activity will use. Field Description Agent Server Specifies the server where the agent containing the script is located. Agent Database Specifies the Notes database file path and name where the agent containing the script is found. The Agent Database browse button located next to the field displays the list of databases available on the server chosen above. The LEI Script Vault database installed by LEI is leivlt6.nsf. Agent Name Specifies the name of the agent containing the script to execute. Punctuation, leading or trailing spaces and so on are not acceptable agent name characters. The Agent button located next to the field displays the list of agents available for the selected database. See the chapter entitled “Introduction to LEI Activities” for descriptions of Activity Execution, Scheduling, and Other options. Agent Development Document You can use the Agent Development document to create a LotusScript agent. You can also use it to perform one of the following actions — simple, formula, LotusScript, imported Java, or Java. 1. To open the Agent Development document, click the Create Agent button in the Scripted Activity Document. If prompted, specify the server name and database name (if you have removed the entries from the Activity Document). The default names are those specified in the Agent Server and Agent Database fields of the Activity document. 2. The specified database (for example the LEI Script Vault database leivlt6.nsf) is opened in Domino Designer. Select the Create Agent button in this view. 288 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide This database is most easily specified using the Agent Server and Agent Database options on the Scripted Activity Document. 3. Enter a name for your new agent and any other dialog box options that are applicable to you. Click the X in the top right of the dialog box to continue. This name can then be used in the Agent Name option on the Scripted Activity Document. Chapter 17: Scripted Activity 289 4. Create your agent syntax in the resultant Agents view. Please see the Notes Help system to specific details about how to perform this task. For more information about creating an agent, see the Notes online documentation. Determining if an Agent is Running To check to see if an agent is running, type the following Show Task command in the Domino server console: sh ta One of the processes listed will be the Agent Manager. It will display as either idle or as the name of the task that it is running. To check the log after the script has run, open the database that it is stored in, select the agent, and from the top menu choose Agent - Log. The log is overwritten each time the agent runs. For each run, the log will state the number of documents selected and the time it started and stopped. 290 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Scripted Activity Considerations This section contains several suggestions for writing Notes agents to be used in LEI Scripted Activities. • The LEI and Notes directories must be listed in your path for LotusScript to execute correctly. • When creating agents, select Shared and specify None as the Target value on the Agents menu. • Provide support for both LEI errors and LotusScript errors. • To test agents using the Notes Integrated Development Environment (IDE), choose “Debug LotusScript” from the File - Tools menu and then run the agent from the menu. You will need either an LEI server, Notes client, or Designer client installation on the local machine. In addition, Notes password entry is not permitted. Either a Notes ID without a password must be used or the option “Share password with Notes add-ins.” Choose the Notes menu option File - Tools - User ID. Chapter 17: Scripted Activity 291 • You can organize your agents within the LEI Script Vault database (leivlt6.nsf), the Notes databases where related data resides, or in some other database. Note Since these agents are “shared,” they will be deleted if the database design is replaced or refreshed. You should disable the design inheritance of the Notes database where you choose to store the agents. • A Scripted Activity cannot open a database that is on a remote iSeries or UNIX server. 292 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Appendix A LEI and Data This appendix provides information about LEI data types and data mapping. Notes and Relational Databases LEI can pass data between data sources, including Notes. It can be helpful in these cases to understand the match-up between Notes objects and database objects. The metadata object of an LEI Activity is a table in a relational database or a Notes form. The following table summarizes LEI terminology in relation to Notes terms and traditional relational database vocabulary. LEI Term Relational DB Term Notes Term Database Database Database (.nsf file) Metadata Table, View, Stored Procedure Form or View Field Column Field or Column Record Row Document or Row Statement SQL query Selection formula LEI Data Types LEI uses the following data types internally when transferring and manipulating data. LEI converts original data types to the closest matching LEI type, retaining information about the original type. When appropriate, LEI can then convert data back to its original type. Data Type Parameters INT Description 4-byte signed integer Maximum value 2147483647.00 Minimum value -2147483647.00 Precision 9 digits Description 8-byte floating point value, corresponding to a C double FLOAT continued 293 Data Type CURRENCY NUMERIC Parameters Maximum positive value 1.7976931348623158e+308 Minimum positive value 0.00 Precision 15 digits Description 8-byte signed integer value: four fixed decimal places Maximum value 922337203685478.00 Minimum value -922337203685478.00 Precision 19 digits Scale 4.00 Description High-precision packed numeric value Maximum positive value 9.99…99e126 Minimum positive value 0.00 Maximum precision 88 digits DATETIME TEXT BINARY Maximum scale 127.00 Minimum scale -127.00 Description 8-byte datetime value with timezone Maximum value 12/31/32767 Minimum value 1/1/1 Accuracy 0.01 seconds Description character-set specific text stream Properties Fixed/variable length, case sensitivity Maximum length 4GB Description Binary stream with optional binary format information Available formats BLOB (unformatted), Composite (Notes Rich Text), Text List (multivalue Text), Number List (multivalue Number), Datetime List (multivalue Datetime), Properties Fixed/variable length, case sensitivity Maximum length 4GB 294 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide The Meaning of NULL LEI regards a NULL as an undefined value, one that cannot be compared to other values. It is not equivalent to an empty string or to an empty value such as zero. A replication will occur, for example, if LEI finds a NULL in a database matching an empty value in the replica. In such cases, use SQL syntax to convert NULLs to values that are equivalent across databases. LEI evaluates the absence of a field in a Notes database as a NULL. Year 2000 Solution LEI uses the Pivot Date solution to the year 2000 date problem. How LEI interprets two-digit dates is dependent on the Century boundary setting in the notes.ini file. The EICenturyBoundary variable can be set to any number between 0-101. If the year is less than the boundary number, then the century 2000 will be used. If greater than or equal to the boundary, the century is 1900. If the boundary number is 101, then LEI uses the current century. The default boundary is 50. Zero always means 1900 and 100 always means 2000. Examples are shown below. How 2-digit years are translated EICenturyBoundary = 0 00 = 1900 55 = 1955 99 = 1999 EICenturyBoundary = 55 00 = 2000 55 = 1955 99 = 1999 EICenturyBoundary = 60 00 = 2000 55 = 2055 99 = 1999 EICenturyBoundary = 100 00 = 2000 55 = 2055 99 = 2099 Appendix A: LEI and Data 295 Appendix B Error Messages and Troubleshooting This appendix provides information for troubleshooting LEI. Installation Troubleshooting See the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for information about troubleshooting the LEI installation. For more information, see the following Web site: http://www.lotus.com/ei Basic LC LSX Errors LCFAIL_MEMORY: Unable to allocate memory. A memory allocation request failed. Either reduce memory requirements or free up system resources. Common causes for this error include extremely large fieldlist record count or fixed stream length. LCFAIL_UNAVAILABLE: Requested functionality is not available. A request cannot be satisfied. This is most commonly returned from a connection which does not support a particular function or operation, for example a read-only connection would return this in response to an Insert operation. LCFAIL_END_OF_DATA: The last data value has been retrieved. This indicates the normal end of processing from any iterative operation, and should not normally be treated as an error condition. All iterative operations, such as fetching from a result set or listing fields, will eventually return this status code indicating that there are no more results available. LCFAIL_INVALID_INDEX: Cannot locate list element. An index parameter provided is beyond the range of the object. For example, requesting the third record from a two-record fieldlist would return this error. 297 LCFAIL_INVALID_LIST: Invalid list direction. An LCLIST enumeration parameter was not valid. One of the LCLIST_XXX constants must be provided for any list parameter. LCFAIL_INVALID_CONVERT: Invalid conversion. Conversion is not supported between the source and target types. For example, conversion between numbers and datetimes will result in this error. LCFAIL_INVALID_TEXT_LIST: This operation requires a valid text list. An LCTextList function requires LCThisTextList to be a stream with a valid text list format (format LCSTREAMFMT_TEXT_LIST and a length of at least two) or a single text value (type LEITYPE_TEXT). Either one of these conditions was not satisfied or the data was not a valid text list. LCFAIL_INVALID_NUMBER_LIST: This operation requires a valid number list. An LCNumberList function requires LCThisNumberList to be a stream with a valid number list format (format LCSTREAMFMT_NUMBER_LIST and a length of at least four). One of these conditions was not satisfied or the data was not a valid number list. LCFAIL_INVALID_DATETIME_LIST: This operation requires a valid datetime list. An LCDatetimeList function requires LCThisDatetimeList to be a stream with a valid datetime list format (format LCSTREAMFMT_DATETIME_LIST and a length of at least 4). One of these conditions was not satisfied or the data was not a valid datetime list. LCFAIL_ZERO_INDEX: All index values are one or greater – an index of zero is not valid. An index parameter provided is zero and all index values for LEI objects are one or greater. LCFAIL_ZERO_COUNT: This operation requires a non-zero count. A count parameter provided is zero, which is not valid in the current context. Most count parameters must be one or greater. LCFAIL_ZERO_OFFSET: All offset values are one-based – an offset of zero is not valid. An offset parameter provided is zero, and all offset values for LEI objects are one or greater. 298 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LCFAIL_ZERO_FORMAT: This operation requires a non-zero stream format. A stream format parameter provided is zero, which is not valid in the current context. Most, but not all, stream format parameters must be a valid LCSTREAMFMT_XXX constant. LCFAIL_NULL_BUFFER: A NULL buffer was provided when one was required. A NULL pointer was provided for a memory buffer parameter, which is not valid in the current context. Some buffer parameters (usually input buffers) must be valid pointers. LCFAIL_NULL_RESULT: A return parameter is required, but none was provided. A NULL pointer was provided for an output parameter. Some output parameters, such as created objects, require valid pointers. LCFAIL_FIXED_LENGTH: A fixed-length stream requires a non-zero length. An LCSTREAM object with the flag LCSTREAMF_FIXED allocates a fixedlength buffer at stream creation time. The length is determined by the stream’s maximum length property, which cannot be zero for a fixed-length stream. LCFAIL_INVALID_FLAGS: The supplied flags are invalid, possibly due to a conflict. A flags parameter contains invalid or conflicting flags. A few flags constants cannot be used together (for example, LCFIELDF_KEY_GT and LCFIELDF_KEY_LT). LCFAIL_TEXT_TRANSLATE: Text translation failure. Translation between text stream formats failed. LCFAIL_NULL_FIELDNAME: A NULL field name was provided. A NULL pointer was provided for a field name parameter. A valid pointer is required. LCFAIL_INVALID_FIELDLIST: Invalid fieldlist. An invalid LCFIELDLIST object instance/handle was used. LCFAIL_INVALID_CONNECT: Invalid connect. An invalid LCCONNECT object instance/handle was used. Appendix B: Error Messages and Troubleshooting 299 LCFAIL_EMPTY_FIELDLIST: This operation cannot be performed on a fieldlist with no fields. A fieldlist with no fields was used in an operation that is invalid on an empty fieldlist. Add one or more fields to the fieldlist and retry the operation. LCFAIL_NAME_FIELDLIST: This operation cannot be performed on a name-only fieldlist. A fieldlist allocated with a record count of zero contains field names only and no field data. Such a fieldlist cannot be used in the current context. Use a fieldlist allocated with a record count of one or greater. LCFAIL_FIELDLIST_REF: Cannot alter fields in a multiply-referenced fieldlist. A fieldlist created as a reference copy of another fieldlist shares field data space with the original fieldlist. Fields cannot be added to or removed from such a fieldlist as long as one copy still exists. LCFAIL_RECORD_INDEX: An invalid fieldlist record index was encountered. An lcRecordIndex parameter was invalid for the corresponding fieldlist. A record index must be no greater than the number of fields in the fieldlist. LCFAIL_RECORD_COUNT: Request to transfer more records than allocated in fieldlist. An lcRecordCount parameter was invalid for the corresponding fieldlist. A record count must be no greater than the number of fields in the fieldlist, less the record index. LCFAIL_LIST_SETUP: Fieldlist iteration requires initial setup. Iterating through fields in a fieldlist with LCFieldlistList requires that LCFieldlistListSetup be called first to prepare the fieldlist for iteration. When LCFieldlistList is called before LCFieldlistListSetup for a particular fieldlist instance, this error is returned. LCFAIL_NO_MERGE_DATA: The data fieldlist in a merge cannot be a name-only fieldlist. A fieldlist allocated with a record count of zero contains field names only and no field data. Such a fieldlist cannot be used as the LCDataFieldlist parameter for an LCFieldlistMerge or LCFieldlistMergeVirtual operation. 300 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LCFAIL_NO_RESULTSET: This operation requires an active result set. The requested operation cannot be performed against a connection without an active result set. A result set must first be created in the connection with LCConnectExecute, LCConnectSelect, or LCConnectCatalog. LCFAIL_NO_WRITEBACK: This operation requires an active writeback result set. The requested operation cannot be performed against a connection without an active writeback context. A writeback result set (WRITEBACK property set to TRUE) must first be created in the connection and a successful Fetch operation performed before a writeback Update or Remove operation is valid. LCFAIL_WRITEBACK_COUNT: Writeback operation record counts must be one. Writeback updates, and removes operate on the current record in a connection’s result set. For these operations, the lcRecordCount parameter must be one. LCFAIL_STATIC_PROPERTY: Static activity properties cannot be used with unnamed activities. Activities initiated with LCActivityAlloc and no activity name exist outside the context of the LEI Administrator. Since static properties are stored within the activity document in the Administrator, they are not valid for unnamed activities. LCFAIL_LCXINIT: LEI system has not been initialized. The LEI API system must be initialized before calling any LEI functions. The system is initialized by command-line execution of an activity or calling LCActivityAlloc. This error is returned when LEI API functions are called before the system is initialized. LCFAIL_ACTIVITY_NOT_INIT: The activity must be initialized before performing any operation. The activity context must be initialized before calling any LCActivity functions. The activity context is initialized by command-line execution of an activity or calling LCActivityAlloc. This error is returned when activity functions are called before the activity context is initialized. LCFAIL_ACTRUN_TIMEOUT: Timeout while running synchronous activities. When executing synchronous activities via LCActivityRunActivity and using a timeout, this error is returned if the timeout occurs before all synchronous activities and their subordinate activities have completed. Appendix B: Error Messages and Troubleshooting 301 LCFAIL_NOT_CONNECTED: This operation requires a connection to a Connector. This error is returned by LCConnect functions which require a successful call to LCConnectConnect. All functions which manipulate or retrieve data require a successful connection. LCFAIL_CONNECTED: This operation cannot be performed with a valid Connector connection. This error is returned by Connectors which require a particular operation to be performed before establishing a connection with LCConnectConnect. LCFAIL_EXTERNAL: External error. This error indicates that an error external to LEI has been produced. The specific external error code and error text are usually available to provide additional information. LCFAIL_ACTIVE_SUBCONNECTION The subconnection (connection property) of a metaconnection can only be set once. This error occurs when setting the property, but a valid setting has already been made. Basic LC LSX Events (Non-Error) LCEVENT_EXTERNAL: External event This status code indicates that an event external to LEI has been produced. This is an informational message. The specific external event code and event text are usually available to provide additional information. Extended LC LSX Errors LCFAIL_INVALID_METADATA: Metadata object <metadata> does not exist. The metadata object to use is provided in the METADATA property. The current value for this property does not represent a valid metadata object in this connection. LCFAIL_TYPE_MISMATCH: Type mismatch for field <fieldname>; LEI: <type>, Connector: <type>. When matching LEI fields to connection fields, the basic type class must match. They must both be numbers, datetimes, or streams. This error is returned when there is a data type mismatch. LCFAIL_DUPLICATE: Duplicate object <name>. An attempt to create an object failed since an object of the same name already exists. 302 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LCFAIL_FIELD_COUNT_MISMATCH: Field count mismatch; LEI: <count>, Connector: <count>. When mapping fields, mismatched fields are only valid in certain circumstances. In general, if there are more source fields than target fields, this error is returned. LCFAIL_KEY_COUNT_MISMATCH: Key count mismatch; LEI: <count>, Connector: <count>. When using key fields, the same number of field names must be provided. LCFAIL_STAMPFIELD_TYPE: Timestamp field <fieldname> must be type datetime; Actual: <type>. When providing a timestamp field, the field must be of a datetimecompatible data type. LCFAIL_FIELD_TYPE: Type mismatch for field <fieldname>used in this context; Expected: <type>, Actual: <type>. Certain operations expect certain data types. This error is returned when a field is used for a particular purpose, but does not have the proper data type. LCFAIL_MERGE_FIELD: Field mapping failed due to a missing field <fieldname>. When mapping fields, a valid match must exist for all required fields. If a field does not have a corresponding match, this error is returned. LCFAIL_MISSING_PROPERTY: No value supplied for required property <property>. A particular property must be set before the requested operation can be performed. For example, the INDEX property must be provided before attempting to create an index. LCFAIL_PROPERTY_CONFLICT: Conflicting values for properties <property> and <property>. Values for two properties conflict. In some cases, properties are exclusive or interrelated, and certain combinations are not allowed. See the relevant connection or activity documentation for more information. LCFAIL_INVALID_PROPERTY: Invalid property <property>. A property not supported by the connection or activity was referenced. Appendix B: Error Messages and Troubleshooting 303 LCFAIL_PROPERTY_VALUE: Invalid property value for property <property>. A property value is not valid. See the relevant connection or activity documentation for more information. LCFAIL_INVALID_CHARSET The character set indicator is not a valid representation. Valid character set representations are any LEI supported character set suffix, listed in the “Character Sets” appendix of the Lotus Connector LotusScript Extensions Guide. For example, for LCSTREAMFMT_IBMCP850, use IBMCP850. LCFAIL_READ_ONLY_PROPERTY An attempt was made to set a read-only property - the connection does not support assignment of the property, only access. Extended LC LSX Events (Non-Error) LCEVENT_CHARACTER_SET: Unsupported connection character set <charset>; using native character set. This status code indicates that a connection’s character set could not be matched to any available LEI character set. This is usually not an error situation, but rather an informational message. The local machine’s native character set will be used. Extended LC LSX Fieldname Errors LCFAIL_OVERFLOW: Data overflow in field <fieldname>. A data overflow occurred when transferring data to or from a field. For numbers, an overflow indicates a value with a magnitude too large. For datetimes, an overflow indicates a year beyond the supported range. For streams, an overflow indicates a value too long. To suppress an overflow error, use the field flag LCFIELDF_TRUNC_DATA or the stream flag LCSTREAMF_TRUNCATE. LCFAIL_PRECISION_LOSS: Data precision loss in field <fieldname>. Precision loss was detected during type-checking. For efficiency, precision loss is only checked during type-checking, To suppress a precision loss error, use the field flag LCFIELDF_TRUNC_PREC. LCFAIL_INVALID_INT: Invalid integer value in field <fieldname>. An invalid integer reference, value or text representation was encountered. LCFAIL_INVALID_FLOAT: Invalid float value in field <fieldname>. An invalid float reference, value or text representation was encountered. 304 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide LCFAIL_INVALID_CURRENCY: Invalid currency value in field <fieldname>. An invalid currency reference, value or text representation was encountered. LCFAIL_INVALID_NUMERIC: Invalid numeric value in field <fieldname>. An invalid numeric reference, value or text representation was encountered. LCFAIL_INVALID_DATETIME: Invalid datetime value in field <fieldname>. An invalid datetime reference, value or text representation was encountered. LCFAIL_INVALID_STREAM: Invalid stream value in field <fieldname>. An invalid stream reference or value was encountered. LCFAIL_INVALID_FIELD: Invalid field <fieldname>. An invalid LCFIELD object instance/handle was used. LCFAIL_INVALID_TYPE: Invalid data type for field <fieldname>. An invalid type constant was used. Only valid LEITYPE_XXX constants may be used to represent a data type. LCFAIL_INVALID_KEY: Invalid key field <fieldname>. A key field name provided does not correspond to a valid field. Key fieldnames must exist in the relevant metadata. LCFAIL_DUPLICATE_KEY: Duplicate key field <fieldname>. A key field name was used twice in the same key list. Remove the duplicate reference. LCFAIL_INVALID_STAMPFIELD: Invalid timestamp field <fieldname>. A timestamp field name provided does not correspond to a valid field. timestamp fieldnames must exist in the relevant metadata. LCFAIL_INVALID_FIELDNAME: Field name <fieldname> is not valid in this context. A field name provided is not valid in the current context. In some cases, field names are restricted in their usage for a particular connection or activity. Appendix B: Error Messages and Troubleshooting 305 LCFAIL_VIRTUAL_FIELD: Unsupported virtual field <fieldname>. A virtual field was found, but the field name is not valid for the connection. See the Connector documentation for a list of valid virtual fields for a particular connection. LCFAIL_VIRTUAL_VALUE: Invalid data value for virtual field <fieldname>. Some virtual fields place restrictions on valid field values. See the Connector documentation for additional information on virtual field values for a particular connection. LCFAIL_INVALID_ORDER: Invalid ordering field <fieldname> An ordering field name provided does not correspond to a valid field. Ordering fieldnames must exist in the relevant metadata. Other General LEI Messages Monitor failure — Error: This operation requires an active result set, Connector Notes Method - SetProperty. This message indicates that LEI is attempting to open a document created as a Notes to Notes Advanced RealTime Activity. However, Notes to Notes Advanced RealTime is not supported. Specifically, in a Notes Advanced RealTime Activity a Notes database cannot be the external system database. Cannot use a Notes connection to a local database — Error: No such database exists: The activity attempting to browse metadata in a connection to a local database cannot locate that database. This occurs when the LEI Administrator hosted by the LEI server does not reside on the Domino server. 306 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Activity Logs The activity log documents contain some hidden fields which may be useful. The documentation of these fields is not meant to be a commitment by Lotus software to keep this format consistent for the future, and these fields may not be compatible with previous or future LEI versions. Field “Error”: When an activity ends in an error state, this field contains a non-zero number that indicates the LEI error code representing the initial error that placed the activity into an error state. When this field is zero or not present, the activity ended in a non-error state (either successfully, or with all errors explicitly cleared). Field “CountList”: This is a list of numbers that indicate the counts of internal connector function calls and record transfers. The entries in this number list are as follows: Number of calls to Connect Number of calls to Disconnect Number of calls to Execute Number of calls to Select Number of calls to Fetch Number of calls to Insert Number of calls to Update Number of calls to Remove Number of calls to Action Number of calls to Catalog Number of calls to Create Number of calls to Drop *Number of records returned through Execute *Number of records returned through Select *Number of records Fetched *Number of records Inserted *Number of records Updated *Number of records Removed *Number of records returned through Catalog * — Indicates that when the number count cannot be determined, the value 4294967295 will be used instead, meaning indeterminate. Appendix B: Error Messages and Troubleshooting 307 Troubleshooting LEI on iSeries This section addresses some common problems of using LEI on iSeries. Below are some common error messages and solutions you may encounter when running an LEI activity. Error in Log Possible Cause Error: Token was not valid. … (-104) Non-display characters Check SQL statement. in SQL statement This error is common if you cut and pasted SQL statement into activity from another source. Note that you can now provide CR/LF characters to improve readability. If the SQL statement is correct, try retyping the statement in the activity. You may also want to use the DB2 connection option for providing trace data in the activity log. This may pinpoint what is objectionable about the SQL statement. Activity Ended With An Error Possible Solution Error: Character conversion cannot be performed. This message means the Consider whether or not you need to LEI job’s CCSID is not run your Domino server under a compatible with the different LOCALE setting. CCSID of the character data in the relational table being reference. For example you could have a job CCSID of 37 and source data CCSID could be 937 (a double byte ccsid). Error: Authorization failed on distributed database connection attempt. This message is most likely because the supplied user name is not correct or the provided password is not correct. Note that the password for data sources can be case sensitive. If going to a remote DB2/400 data source, make sure the password is UPPER case. If going to any other DB2 data source, make sure the password reflects the case sensitivity required on that target platform. continued 308 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Error in Log Possible Cause Possible Solution Error: Relational database <database name> is not in relational database directory. Either the database name provided in the DB2 connection document is not specified correctly (note no longer case sensitive) or the named database is not registered in the RDB directory. Do a WRKRDBDIRE CL command and determine if the target database is registered. If not, add the database and the necessary connectivity information to that data source. Note that if you want to connect to the local DB2/400, a named entry for the *LOCAL database must be supplied. Error: <table name> in <library name> not valid for operation. Function -Execute- (-7008) Most likely reason is that the Journal option is not correct in the connection or the DB2 table required to be journalled. If the DB2 table is not journalled on iSeries, verify that the DB2 connection you are using has the non-journal option enabled. The default setting for all LEI activities is to run with commitment control. Journalling is required to run under See SQL7008 in the QSQLMSG message file commitment control. You must check the non-journal option to turn for other possibilities. off this transaction dependency. Finding iSeries Job Logs The primary source of information about a failing LEI server or activity is the LEI log database. As LEI is a server addin, you may also need to check the Domino server console for any Domino related diagnostics. Occasionally you may need to find an iSeries system job log. Note that all LEI activities run under the user profile QNOTES on the iSeries. Enter the following command to find the job log: WRKSPLF QNOTES If there is no job log, you may need to change the Job Description for the executing Domino server to generate a job log and then reproduce the failing request. To do this, find the job description associated with the subsystem of your executing Domino server. For example, if your Domino server runs under subsystem DOMINO01, then the job description associated with that Domino server is DOMINO01 (*JOBD) in library QUSRNOTES). Enter the following command: CHGJOBD QUSRNOTES/DOMINO01 LOG(4 10 *SECLVL) Appendix B: Error Messages and Troubleshooting 309 The following are LEI-specific job names: • LEI • LEICSM • LEIACT The LEI job controls the overall LEI server, the LEICSM job communicates with the LEI control store (the Administrator databases) and the LEIACT job is the process that actually runs the activity. In addition LEI agents can run under HTTP and Advanced RealTime runs under SERVER or HTTP. There is another set of jobs that also factor into LEI running DB2-connected Activities on iSeries. These are the QSQSRVR jobs. These are pre-started jobs in the QSYSWRK subsystem that perform DB2 access in a thread safe environment on behalf of the activity job (LEIACT), the real time job (SERVER or HTTP) or the LEI agent job (AMgr). For each DB2 connection established by the LEIACT, a corresponding QSQSRVR job in the QSYSWRK subsystem will be allocated to that connection and will perform all the DB2 SQL requests on behalf of the activity job. It may be necessary at times to find a QSQSRVR job log for problem determination. These jobs are pre-allocated under the user profile QUSER. It may be necessary to change the job description associated with the QUSER profile in order to generate a job log. Additional Tips and Techniques for using LEI on iSeries Listed below are general tips when using LEI on iSeries. • If you are building an activity that specifies subdirectories within your Domino directory, remember that QNOTES must be authorized to those directories and to use the forward slash (for example demodir/mydir/mydoc.nsf) • Using the Create Metadata option when using a DB2 connection when the DB2 target table does not exist results in a CREATE TABLE for the DB2 target (for activities such as Direct Transfer and Replication). However, a CREATE TABLE done by LEI may not be desirable because LEI text fields are not delimited in size because they are mapped from Notes text fields. When the fields are created in DB2, they can be very large. A CREATE TABLE will expand all undelimited text fields to fill out the maximum allowable relational record. On iSeries, this would be character fields that fill out what remains of a 32766 byte record. Although you can qualify the maximum text field size in the Notes connection options, that text field size would apply to all undelimited text fields. You should create the DB2 table in advance or use a Command Activity that creates the table with more reasonable CHAR and VARCHAR field lengths. 310 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide • Do not set your LEI server polling interval too low - 60 seconds is reasonable; 5 or 10 is not. • It is recommended that for Direct Transfer activities you set the Number of Records to Transfer Concurrently to some number other than 1 for better transfer throughput. Depending on the transfer record size, this might be 20-100. • If you are using replication with timestamp to replicate a DB2/400 table with a Notes database and your DB2/400 table is restored from backup or has had significant changes applied to it, such as through APYJRNCHG, you should select the Reset Replication Timestamp button prior to the next run of the replication activity. Whenever you suspect that the two databases are significantly out of sync due to a system action or utility you should run with the reset option. • Qualifying an LEI DB2/400 request by specifying a particular member of the physical file is not supported. The first member of the specified physical file (table) is always used. • If you are taking the overwrite option on Direct Transfer and clearing out the target first, and that target happens to be a DB2/400 table, you might want to consider using a command activity to clear the physical file member first (CLRPFM) and then do the direct transfer without the overwrite option. This would eliminate the potentially lengthy SQL Delete and the additional journal entries that might occur. You could then schedule these two activities as chained (with direct transfer a dependent activity of the clear request). • To work around this situation, you may need to reorder the fields in your form to reflect the order of the key fields as specified in the “Key Fields” option of your Replication Activity form or vice versa. Also, you should not use a full timestamp field as a key field. DB2/400 precision is more significant then is Domino’s precision when it comes to timestamps and you will not be guaranteed a unique lookup. • On Direct Transfer, if you specify “Try Update Before Insert”, the key field(s) specified in the “Key Fields for Update” field must reflect the source connection, and not the target connection key field names. • Stored Procedures used on Direct Transfer requests or Advanced RealTime activities must run in the activation group of the caller on iSeries (which is LEI). To determine the activation group state of your program enter the command DSPPGM. If the Activation group attribute is not *CALLER, you will get an abnormal termination in the log database. In addition, the stored procedure must be invoked under the authority of QUOTES user profile. Make sure that the program is made available to public invocations or that QUOTES is authorized. Appendix B: Error Messages and Troubleshooting 311 • With stored procedures to DB2, also note that NUMBER fields from a Notes form will be bound to argument values (in C) of type DOUBLE on a call to a stored procedure. Your stored procedure will need to cast accordingly in order to map to the appropriate SQL type. Advanced RealTime Configuration, Troubleshooting, and Error Messages This section provides information about error messages, and known problems that you may encounter under certain conditions while using Advanced RealTime. For information about notes.ini variables for Advanced RealTime activities, see the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide. Unable to Access Data with Virtual Fields Activities After Upgrading After upgrading from DECS to LEI, if you are unable to get data when running Virtual Fields activities, open each Virtual Fields activity document and ensure that the Domino server specified in the activity document is correct. Security Issues in Advanced RealTime There are two issues regarding security in Advanced RealTime: • Advanced RealTime administrator (Notes browsing and initialize keys) • LC LSX against the Lotus Connector for Notes or the Lotus Connector for File System The Advanced RealTime administrator issue occurs if you attempt to browse Notes when you create an activity or use the Initialize Keys function. This may generate a LotusScript error regarding access to unrestricted agents. Your Advanced RealTime activity must be allowed to run unrestricted agents for these capabilities to work. The LC LSX issue occurs when agents or other LotusScript code, which use the Lotus Connector for Notes or the Lotus Connector for File Systems, must be signed with a Notes ID that has permission to run unrestricted agents. This also applies to any script libraries used by this code. Specifically, connections through these Connectors will be blocked with an error unless the signing Notes ID has permission to run unrestricted agents. 312 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide You can use either of the following solutions to address these issues. • Add the ID “Enterprise Connector Products/Lotus Notes Companion Products” to the “Run Unrestricted Agents” field in the Domino Server’s entry in the Domino Directory (NAB) containing that server. This field is found under the Security tab. • Resign the Advanced RealTime Administrator (LEI) database with a Notes ID that already has permission to run unrestricted agents. Resign the Advanced RealTime Administrator through the Domino Administrator client, not through the normal Notes client. Instruction for configuring your Domino Directory is provided in the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide in the “Setting Access Controls in the Domino Directory” section of each platform-specific chapter. Stored Procedures in Sybase or OLEDB and Advanced RealTime When attempting to create an Advanced RealTime Activity that uses stored procedures in Sybase or OLE DB, you may find that the fields cannot be mapped and the following error displays: Both the Notes and connection data field list must contain at least one value. The procedure can be selected, but the activity cannot be completed or saved. Sybase normally returns a result set from a stored procedure, which an Advanced RealTime Activity “sees” as if it were a result set from a table or view. Because of this, when setting up an activity using a Stored Procedure, you must do the following: 1. Create the Advanced RealTime Sybase Connection Document by selecting Table, not Procedure. 2. Use or create a table containing the exact column names returned by the stored procedure, and use this in the mapping process in the Advanced RealTime Activity. 3. Identify the stored procedure to use in the activity options under the appropriate “When Intercepting a document” section. There are separate sections for the Open, Create, Update, and Delete events and separate stored procedures are needed for each event to perform functions specific to that event. When the Advanced RealTime Activity is activated, the stored procedure is used, overriding the field mapping. Appendix B: Error Messages and Troubleshooting 313 Example: This example illustrates how to configure an Advanced RealTime activity with a Sybase connection using stored procedures. It assumes that you have a Notes form with two fields, Key1and Lastname, and that you have Sybase table(s) with many fields. It also assume that you have a stored procedure which accepts one input parameter and returns a result set with Key1, Lastname columns. Finally, it assumes that you have an activity which returns data based on the Open event. Using this example, you will create a Sybase table, a stored procedure, and an Advanced RealTime Activity. The Sybase Table First, create a Sybase table that has columns made up of what you are ultimately mapping (the input parameter and the result set that your procedure uses). For instance, in this example the dummy table would contain the following columns: Key1 VARCHAR(5) Lastname VARCHAR(25) The Stored Procedure The procedure takes input (the same as what you selected for the Advanced RealTime key), and returns a result set that includes the key and all mapped fields in the Advanced RealTime Activity: CREATE PROCEDURE dbo.proc1 (@Key1 varchar(5) ) Activity. -- Input. This key, matches key in AS BEGIN select Key1, Lastname from debtest of -- Result set. Returns all -- the mapped fields, including key(s). where Key1 = @Key1 END Note If you are working with Sybase, note that Sybase is generally case-sensitive, but the @var (@Key1) in the Proc must have the same name and match case with your Sybase column and Advanced RealTime mapped key. Sybase will accept a different name/case if you execute the procedure 314 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide from an SQL tool, using Execute <Procedure>, but Advanced RealTime needs the name/case to be the same or it will return the error below: 03:48:27 PM ART: Monitor failure -- Error: Procedure proc1 expects parameter @key1, which was not supplied., Connector 'sybase', Method -Call- (201): proctest Note In this example, Table was selected, not Procedure. Procedure is valid only if you are returning all values using output parameters, as is done with Oracle, for example. In the above Virtual Fields Activity, dbo.proctable is a table created specifically with the fields required for mapping. Once the events are selected, specify the procedure to be used under the Options, as shown in dbo.proc1. After inputting the procedure name on the activity document, pressing F9 will refresh and show the input parameter that the procedure requires (Key1). At this point, Virtual Fields returns the values in the mapped fields to the form being monitored. To use other events, create procedures appropriate to the event. Note Since the stored procedure does not return all key values, “Initialize keys” does not work with this functionality; it does not recognize the contents of a table or view. If key documents are required in Notes, create them using an agent or by a separate Virtual Fields Activity that pulls those key values into Notes documents. Advanced RealTime Activity Error Messages Error messages that pertain to Advanced RealTime Activities are listed below. Cannot use field [‘FIELDNAME’] as both a key and a data field Fields provided for Advanced RealTime activities must be either key fields or data fields - a field cannot be used as both in a single Advanced RealTime Activity (although one field can be used as a key in one activity, and data in another activity). Remove any field that is used as both a key field and a data field from at least one of those lists. Failure accessing shared RealTime Activities table An internal error was encountered when attempting to access activity information. Record as much information as possible about the circumstances and contact Lotus software technical support. Appendix B: Error Messages and Troubleshooting 315 Unexpected internal failure in RealTime monitoring An internal error was encountered when attempting to access context information. Record as much information as possible about the circumstances and contact Lotus software technical support. Update of key field [‘FIELDNAME’] is not permitted Key values in a document were altered, but the Advanced RealTime Activity indicated that changes to key fields should be blocked. Updates to both Notes and the external system were aborted. This record has changed in the back-end database since being opened action cancelled The Advanced RealTime Activity option to check the external system for changes before writing changes to the back end was enabled, and the check indicated changes in the back end. Since the document was opened, another system or client changed the corresponding external record. Cannot locate corresponding external record The key values in the opened document being monitored by a Advanced RealTime Activity did not correspond to a record in the external system. This error can be suppressed by selecting the Advanced RealTime Activity option to create the external record if it doesn’t exist - instead of the error, a new record corresponding to the current Notes document data will be created. Failure compiling Filter Formula: FORMULA COMPILATION ERROR The filter formula provided for the Advanced RealTime Activity failed compilation. The compilation error is generated with the error message. Fix or remove the filter formula to successfully run the Advanced RealTime Activity. Failure compiling Pre-Open Formula: FORMULA COMPILATION ERROR The pre-open formula provided for the Advanced RealTime Activity failed compilation. The compilation error is generated with the error message. Fix or remove the pre-open formula to successfully run the Advanced RealTime Activity. Failure compiling Post-Update Formula: FORMULA COMPILATION ERROR The post-update formula provided for the Advanced RealTime Activity failed compilation. The compilation error is generated with the error message. Fix or remove the post-update formula to successfully run the Advanced RealTime Activity. 316 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Failure compiling Post-Create Formula: FORMULA COMPILATION ERROR The post-create formula provided for the Advanced RealTime Activity failed compilation. The compilation error is generated with the error message. Fix or remove the post-create formula to successfully run the Advanced RealTime Activity. Failure compiling Post-Delete Formula: FORMULA COMPILATION ERROR The post-delete formula provided for the Advanced RealTime Activity failed compilation. The compilation error is generated with the error message. Fix or remove the post-delete formula to successfully run the Advanced RealTime Activity. Unknown OS error: libdecsext.* This is a Notes error reporting that Notes couldn’t load the extension manager library. Check that you have properly installed and configured LEI. If problems persist, contact Lotus software technical support. Realtime service addin task initialization failed This indicates that the LEI addin task startup encountered an error. Check that you have properly installed and configured LEI. If problems persist, contact Lotus software technical support. Realtime service is unable to allocate addin task resources The Advanced Realtime service is unable to allocate additional resources. Check that you have properly installed and configured LEI. If problems persist, contact Lotus software technical support. Realtime service cannot connect to external system You may not have the proper connectivity software installed that is required for accessing the external data system. Refer to the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for information about the native software required for connectivity to each of the LEI supported data sources. Realtime service cannot find external table/metadata The metadata selected for this activity does not exist in the external data source. Realtime service cannot find external procedure/transaction The document open event captured by LEI encountered an error. The error details are logged to the Domino server log. Appendix B: Error Messages and Troubleshooting 317 Realtime service error retrieving external record The document open event captured by LEI encountered an error. The error details are logged to the Administrator activity log for this activity. Realtime service error inserting external record The document creation event captured by LEI encountered an error. The error details are logged to the Administrator activity log for this activity. Realtime service error updating external record The document update event captured by LEI encountered an error. The error details are logged to the Administrator activity log for this activity. Realtime service error deleting external record The document deletion event captured by LEI encountered an error. The error details are logged to the Administrator activity log for this activity. Realtime service cannot locate the corresponding record in the external system The document key field values do not correspond to a record in the back end data source. The record in the external data system may have been deleted. Realtime service unable to update document due to key field changes; changes to key fields have been disabled The key fields in the external data have been modified since the document has been opened. To allow key field changes, select the appropriate setting for the option Key Field Updates in the Document Update options section of the Activity Document. Realtime service is unable to update document due to conflict; the external record has been modified since being opened The external data has been modified by another application since the document has been opened. Close and reopen the document. Realtime service error storing external attachment There was a problem storing the attachment portion of the document to the external data store. Realtime service error retrieving external attachment There was a problem retrieving the attachment portion of the document from the external data store. 318 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Realtime service cannot locate the attachment in the external system There was a problem locating the attachment portion of the document in the external data store. Realtime service data overflow accessing external record For document open events, this indicates that data in an external field was too long. Usually this is due to text longer than 64K. To avoid this problem, change the Notes field to Rich Text. (See the error in the Administrator activity log for this activity for information on which field caused the overflow.) For document updates and inserts, this message indicates that the document data overflowed a back end field. You may need to change the data type in the back end to store large amounts of data. Realtime service error deleting external attachment There was a problem deleting the attachment portion of the document from the external data store. Realtime service error browsing the external system There was a problem retrieving information about the names of tables in the database, or retrieving the datatype information about the columns in a particular table. Realtime service encountered a missing or invalid Data Connection Resource You get this message when you are using the Domino Designer Data Connection Resource (DCR). The Data Connection Resource that was referred to by one or more of the fields on the form is not valid, or could not be located. Realtime service error scanning the external system. Some external records may have been ignored During a view refresh, an error occurred when retrieving document information from the external data store. Realtime service error loading external agents/design elements An error occurred while accessing a virtual agent. Appendix B: Error Messages and Troubleshooting 319 This application uses a Realtime Service Activity which is currently not running. Contact your Notes administrator. During an attempt to open or delete a document, a needed running Advanced Realtime activity could not be found. Verify that the activity monitoring this database form is running. If so, fully refresh your view (Shift F9) in the possibility that the external data source record has been deleted. Also check the activity log for any additional error information. 320 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Appendix C Tutorials This chapter provides tutorials for building some of the LEI Activities. You can use these tutorials to learn how to build these LEI Activities. Introduction to the Tutorials The examples in this appendix guide you through the process of building several LEI Activities, including the connections that they use. The examples use sample Notes databases supplied with LEI. The examples use an existing Notes database as the source connection. The target connection can use any Lotus Connector in your environment that you choose. LEI will create new metadata objects (tables or Notes forms) when transferring or replicating data from the Notes database to the target database. The tutorials in this appendix address the following tasks: • Building a Direct Transfer Activity Build an activity that transfers data from a sample Notes database to a database of your choice. • Building a Polling Activity Build an activity that polls a sample Notes database for a condition and then triggers a direct transfer Activity. • Building a Replication Activity Build an activity that replicates a sample Notes database to a database of your choice. • Building a Scripted Activity Build an activity that employs the two agents supplied in the LEI Script Vault (leivlt6.nsf) in conjunction with the supplied leipackagetrack.nsf sample database. Build an activity to execute LotusScript commands for greater functionality with your data transfers. 321 • Building an Advanced RealTime Activity Build an activity that monitors external system data in real time through a Notes application. Build an activity that queries dynamically using your Notes or Web browser. • Additional Advanced RealTime Examples Build a Virtual Fields or Virtual Documents activity that uses Virtual Attachments. Using filter formulas, storage options, monitor orders, and stored procedures in Virtual Fields activities. After building these examples, you will have scheduled Activities that could continue processing. You may want to delete or disable the Activity schedules when you’re done with the examples. Assumptions The tutorials assume that you have a working knowledge of Lotus Notes. If needed, you can refer to the Notes online Help for assistance. These tutorials also assume that you have installed LEI and all the connectivity software required for your system data sources. Finally, the tutorials work in a linear fashion, building on activities and connections created at the start of this appendix. Before Beginning Before beginning, you will need to have the LEI Administrator and log databases on your Notes client desktop. You will also need to know on which server the LEI sample databases leiempsamp.nsf and leipackagetrack.nsf are located, as well as the LEI Script Vault (leivlt6.nsf). These databases are supplied as part of the LEI installation. They will be on the LEI server machine and will need to be copied to a Notes server. These databases should also be added to your Notes desktop, since you will modify them in these tutorial exercises. Be sure that the LEI server is running and can communicate with all external data sources that you intend to use. See the IBM Lotus Enterprise Integrator for Domino (LEI) Installation Guide for information about connectivity requirements and testing connectivity to your data sources. 322 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Building a Direct Transfer Activity This section describes the steps in creating a Direct Transfer Activity. It describes the general procedure for setting up and executing a transfer of data from a Notes database to another database. The main steps in performing a Direct Transfer Activity are listed below: 1. Create the Connections. 2. Create the Direct Transfer Activity. 3. Run the Activity. 4. View the Log. Step 1: Create the Connection Documents Before you create an Activity, you must create the Connections that the Activity will use. A Connection defines access to a database that LEI will interact with. A Direct Transfer Activity requires a source database connection and a destination database connection. Note You should have already established connectivity to any database you intend to use. The Lotus Connectors and Connectivity Guide describes that process. Creating the Source Connection In this example, the source Connection is the sample Notes database leiempsamp.nsf, which is located in the help directory. To create the source Connection, follow these steps: 1. Start Lotus Notes and open the LEI Administrator database decsadm.nsf. 2. From the LEI Administrator click Add Connection — Connection to Notes. 3. In the Name field, enter a descriptive name that you can use later to identify the Connection in a list. You’ll use this name to pick the Connection in the course of creating the Direct Transfer Activity. For the purposes of this example, enter Sample 1 Source. 4. Enter the name of the Notes server on which the Notes database leiempsamp.nsf is located. Leave this field blank if the database is local. 5. Enter the Notes database name leiempsamp.nsf. Appendix C: Tutorials 323 6. Enter the Category name of Samples. By assigning this category to your sample documents, you will find it easier to manage those documents as a set. This is a practice you may find helpful when using LEI. 7. Save and exit the document. Creating the Destination Connection The Destination Connection Document that you create depends on the database that you choose for the destination. To create the Connection, complete the following steps: 1. From the LEI Administrator, click Add Connection. Choose the destination database type from the resulting list. Note This is the target database in which LEI will create a metadata to receive the data. 2. In the Name field, enter Sample 1 Destination. 3. Complete the remainder of the document as needed. You must enter the location of the database and may need to enter a user name and password for access. 4. Enter the Category name of Samples. 5. Save and exit the document. Now that you’ve created the two Connection documents, you can examine the Connections view to make sure that they were created. Viewing the Connections The Activities and Connections view displays your activities and Connections by Name. It can also display names by category. To view the Connections by category, do the following: 1. In the Navigator bar of the LEI Administrator, expand the Activities by Type option. Click on the By Category option and expand the Samples category type. 2. Expand the Link subcategory until you see the Sample 1 Source and Sample 1 Destination connections you created. Note To edit those Connections, you can double-click the document name in the view and then enter edit mode by pressing Ctrl+E or double-clicking within the document. 324 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 2: Create the Direct Transfer Activity After defining the Source and Target Connections, you can create the Direct Transfer Activity to transfer data from one database to the other. Complete the steps below to create the Direct Transfer Activity. 1. From the LEI Administrator, click Add Activity and select Direct Transfer from the resulting list. 2. Enter the Activity name Sample Transfer in the Name field. 3. Identify the Source and Target Connections: • Use the drop down button for a list of the available Connections or press Enter with the cursor in the Connection Name field. • From the list of Connections, select Sample 1 Source. • Repeat for the Connection Name field under Target, selecting Sample 1 Destination. 4. In the Form field under Source, enter the Notes form Emps. 5. In the Table field under Target, enter the table name, Employees. 6. The Select Statement field specifies the data you want to transfer. The statement can be a SQL query or, for a Notes source database, a selection formula. For this example, enter the following Notes formula: SELECT @ALL This formula will create a result set identical to that of the set of documents associated with the Notes form Emps. 7. In the Field Mapping Section, choose Automatic and verify that By Name is selected. 8. Click the Target Data Options tab. 9. In the Existing Data Options section, enable the Create Target Metadata option to create a new table for the transferred data. Leave the other options as they are. 10. Enter the Category name of Samples. 11. Save and exit the document. This completes the process of creating a Direct Transfer Activity. Because the Activity’s schedule was not enabled when you closed the document, you’ll have to manually launch it. Appendix C: Tutorials 325 Step 3: Run the Activity Next, you’ll launch the Activity manually and use the Active view to display its status. To launch the Activity and view the list of running Activities, do the following: 1. In the LEI Administrator, expand the Activities by Type option. Click on the By Category option and expand the Samples category type in the view. 2. Expand the Activity subcategory until you see the Activity you created. 3. Highlight the name of the Activity that you just created, Sample Transfer, and click Start. You can alternatively click Start at the top of the actual Activity Document to execute the Activity. Viewing the Activity Status There are three main ways to display an Activity status. • Click the Activities by Type option in the LEI Administrator Navigator panel. All existing Activity names display, along with their current status. • Expand the Activities By Type option in the LEI Administrator Navigator panel. Click Data Management to display a more detailed list of all existing Activities and their current status. • Expand the Activities By Type option in the LEI Administrator Navigator panel. Click By Category. Click a category type, for example Direct Transfer, to display the current status of only the Direct Transfer Activities. If you do not see a specific Activity listed, press F9 periodically until that Activity name appears as a scheduled job. If you still do not see the Activity listed, it may have already completed. You can view the Log to learn about Activities that have already run. 326 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 4: View the Log The LEI log documents the process and outcome of each Activity. There are three ways to view the log: • Click the View Log button on the individual Activity Document • Highlight an Activity in the Activity view and click the Log button in the action bar • Click the Log option on the LEI Administrator Navigator Note If you open the Log from the Activity Document, LEI displays the Log entry describing the latest execution of the Activity. For this example, once the Activity has completed, highlight the Sample Transfer Direct Transfer Activity in the Activities view and click the Log button in the action bar above the Activities view. Building a Polling Activity The steps described here give you the overall procedure for polling. Polling is a way of initiating one or more Activities based on a condition. Once the Polling Activity is running, it checks a selected database at regular specified intervals looking for a particular condition. When the condition returns true, the Polling Activity executes one or more other Activities. The Polling Activity can be thought of as a trigger for other Activities. In this example, the Polling Activity will trigger a Direct Transfer Activity. Step 1: Create the Polling Activity The Polling Activity requires a Connection identifying the database to be polled. Instead of creating a new Connection for the purposes of this example, we’ll use the existing Connection Sample 1 Source. You’ll schedule the Polling Activity to run between 8 AM and 5 PM each day. During that period, it will poll the database once a minute. Complete the following steps to create the Polling Activity. 1. From the LEI Administrator, click Add Activity. Select Polling from the resulting list. 2. Enter the Activity name Sample Polling in the Name field. Appendix C: Tutorials 327 3. Identify the database to be polled by selecting the connection named Sample 1 Source. Note This connection was created in the previous section entitled, “Building a Direct Transfer Activity”. If you have not already created this connection, do so now following the instructions in the previous tutorial of this appendix. 4. In the Table or Form Name area just under the Connection Name field, enter Emps. 5. Enter the Trigger Statement SELECT HIREDATE=@TODAY. Currently no record satisfies this condition, but you’ll add such a record once the Activity is created. 6. Leave Polling Frequency unchanged so that the polling occurs every 60 seconds. Note This frequency value does not determine when the Polling Activity actually runs. As with other Activities, that is determined by the schedule. 7. At Activities to Execute, select the Activity named Sample Transfer. This is the Activity that will be executed when the Command Statement returns a true condition (for example, once you add a new record that meets the condition). Note This Activity was created in the previous section entitled, “Building a Direct Transfer Activity”. If you have not already created this Activity, do so now following the instructions in the previous tutorial of this appendix. 8. For this example, leave the Reset Trigger information as is. 9. Click the Error Handling Options tab and enter the Maximum Event Count of 1. Leave other options unchanged. Note This value specifies the maximum number of times that Activities will be executed as a result of a true condition before the Activity terminates. Each true condition and associated execution counts as one event. Zero sets no limit on the number of times that events can occur. 10. Click the Activity Execution Options tab to optionally designate an LEI server on which to execute the Activity. Leave other options unchanged. 328 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 11. Click the Scheduling tab and set the Schedule field to Schedule Enabled. Note For a Polling Activity, scheduling behavior is slightly different than with other Activities. Other Activities conclude once they complete their processing. Unless otherwise configured, however, a Polling Activity keeps running (and polling) once it’s initiated. As a result, the Repeat Interval (how often the Activity is initiated) need not be more frequent than the window of time within which the Activity can run. That window of time, in this case, will be defined in the next step as being from 8 to 5 daily. 12. By default, Repeat Interval is set to every 60 minutes. Change the Repeat Interval to every 1 (one) Day. 13. In the Run at Times field, enter 8:00 AM - 5:00 PM to have the Activity execute within that range of hours. Given the Repeat Interval of one day, the Activity will start polling at 8:00 AM and stop polling at 5:00 PM. According to the Polling Frequency defined earlier, it will poll every minute within that period of time. Note If you are creating this example outside that time, use a range that includes the current time. 14. Enter the Category name of Samples. Appendix C: Tutorials 329 15. Save and exit the document. 16. View the Samples Polling Activity using the By Category display option. In the next section, you’ll create a record that will cause the polling condition to become true and queue Sample Transfer to be run. Step 2: Cause the Polling Condition to be True To create a true condition and cause the Polling Activity to schedule the subordinate Activity, do the following: 1. Open the supplied sample database leiempsamp.nsf, located in the help directory. 2. Create a new document in the sample database by selecting Create Emps from the Notes menu. 3. Enter the following information in the form: EMPNO 7790 ENAME NNOTES JOB ENGINEER MGR 7791 HIREDATE 9/12/01 SAL 1000 (enter today’s date) COMM DEPTNO 72 Note This document will satisfy the polling condition: SELECT HIREDATE = @TODAY. 330 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 4. Save and close the form. 5. Start the Activity. 6. Check the LEI Administrator to see that the Activity correctly initiated Sample Transfer. Check the log for the status of the Activity. Note The Polling Activity will continue until it is manually disabled by stopping the activity manually and disabling the schedule. Step 3: Clean up the Samples After trying out these examples you can delete the Connections and Activities. Alternatively, you can disable the Activities, which will allow you to reuse them later on. Deleting the Samples Category Connection and Activity documents 1. In the LEI Administrator, display By Category. 2. Click the Samples arrow to display it contents. Continue displaying its subcategories until the names of the sample activities appear. 3. Select the documents to delete and press the Delete key or Ctrl+X on your keyboard. 4. The documents will be deleted when you leave the view. Disabling the Sample Activity Documents 1. In the LEI Administrator, display By Category. 2. Click the Samples arrow to display it contents. Continue displaying its subcategories until the names of the sample activities appear. 3. Select the first Activity document to disable. 4. Press Ctrl+E to edit the document. 5. Under the Scheduling tab, change the Schedule field to Schedule Enabled. 6. Save and close the Activity. Appendix C: Tutorials 331 Building a Replication Activity These steps described the overall procedure for configuring and executing database replication. The databases used are the LEI sample leiempsamp.nsf and a second database of your choice. The type of replication shown in this example is called “primary key”. It is a method of replication that compares one or more key fields in the replicating database (Source) and the one to be replicated (Target). By comparing key fields, LEI matches records and determines whether replication requires a record update, insert, or delete operation. An update occurs when the primary keys of both records are identical (assuming the data in the other fields is not), an insert occurs when the primary key exists only in the source, and a delete when it exists only in the target. Step 1: Prepare the Database In this example, you will use the same database as in the preceding tutorial (entitled “Building a Polling Activity).” By adding a document to the sample Notes database, you will be able to see how the replication works. The record will be replicated to the other database. 1. Use the Notes desktop to open the supplied sample database leiempsamp.nsf. 2. Create a new document in the sample database by selecting Create Emps from the Notes menu. 332 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 3. Enter the following information in the form: EMPNO 6627 ENAME JNOTES JOB ENGINEER MGR 6628 HIREDATE Today’s date SAL 1000 COMM 0.05 DEPTNO 70 4. Save and close the form. Step 2: Create the Connections Create a Notes connection and a second connection to a database in your environment. You will identify the source database (the one against which the other database is replicated) when you build the Replication Activity. Creating the Source Connection This first connection will identify the Source database. To create the connection complete the following steps: 1. From the LEI Administrator, click Add Connection - Connection to Notes . 2. In the Name field, enter Sample 2 Master. 3. Enter the name of the Notes server where the sample database is located. 4. Enter the name of the sample Notes database leiempsamp.nsf, which is located in the help directory. 5. In the Category field, enter Samples. Appendix C: Tutorials 333 6. Leave all other options set to their default values. 7. Save and close the document. Creating the Target Connection This second connection will identify the target database. In this example, the data in the target will be updated as a result of the replication. To create the connection complete the following steps: 1. From the LEI Administrator, click Add Connection. 2. Select the database type that you want to use from the resulting list and click OK. 3. In the Name field, enter Sample 2 Updated DB. 4. Enter the name of the database that you want to have updated based on the information in the Source database. For this example, this database will be named CurrentEmployeeData.nsf. This database was created as a copy of the supplied sample database leiempsamp.nsf and resides on the local LEI server in this example. 5. In the Category field, enter Samples. 334 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 6. Leave all other options set to their default values. 7. Save and close the document. Step 3: Create the Replication Activity Once the two connections are created, create the Activity that will replicate the source database. Data is replicated at the table level (or at the Notes form level). In this example, you will schedule the Activity to run every hour and restrict the range of time within which it can run. Complete the steps below to create the Replication Activity. 1. From the LEI Administrator, click Add Activity. Select Replication from the resulting list. 2. In the Name field, enter Sample Replication. Appendix C: Tutorials 335 3. In the Source Connection field, specify Sample 2 Master. 4. In the Target Connection field, specify Sample 2 Updated DB. 5. Enter the Form Name for both Connections. Enter Emps for the Source Connection and Sample1 for the Destination connection. 6. Enable Automatic Field Mapping. 7. For the Source Key value, enter Empno. 8. For the Target Key value, enter Empno. 9. Enable the Mapping by Name option. 10. If the databases that you identify for replication keep the date and time of each change to a record in a timestamp field, you can identify those fields and do primary key/timestamp replication. Leave Timestamp Replication deselected for this example. 11. Click the Data Options tab and then select Reduced-precision Comparison. Note If the source database uses a different precision than the target database for datetime or floating point numbers, false mismatches may occur. This option prevents that by using a lower common degree of precision. Datetime values are compared to the second and floating point numbers to ten digits of precision. 12. The Conditional Clause for the Source Connection could be a Notes formula or SQL WHERE clause to further limit the result set. Leave this field blank for this example. For this example, do not enter a condition clause. 13. Optionally click the Activity Execution Options tab to specify an LEI server on which to execute the Activity. Otherwise leave these options set to their default values. 14. Click the Scheduling tab and then select Run Activity Once and Disable. This option only allows the replication to run just once as a trial, after which its schedule is automatically disabled. 15. Change the Schedule field to Schedule Enabled. 336 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 16. Limit the times and days on which the Activity can run. For this example, do the following: • In the Run at Times field, enter 7:00 AM - 5:00 PM to have the Activity execute only within that range of hours. • Schedule the replication to run every hour. Enter 60 in the Repeat Interval field and select Minutes. • In the Days of Week field, delete today’s day name and tomorrow’s by clicking the down arrow and then deselecting the appropriate days of the week. With these settings, the Activity will not run until 7:00 AM on the day after tomorrow. 17. In the Category field, enter Samples. 18. Leave all other options set to their default values. 19. Save and close the document. Appendix C: Tutorials 337 Checking Activity Status and Running the Activity Check the status of the Activity in the LEI Administrator. 1. In the LEI Administrator, choose Active from the Navigator. 2. Locate the Activity you just built, Sample Replication. 3. Select the Activity. 4. Select Start from the LEI Administrator. Note To test the Activity and your scheduling specifications, you can select Trial Run Replication, within the Activity Document, to perform a test run prior to actually executing the Activity. 5. Periodically press F9 to refresh the view until the Activity completes. Note After the Activity completes, it will be scheduled for 7:00 AM the day after tomorrow (in this example on Thursday). The Run Activity Once and Disable option only pertains to a scheduled run. 6. Open the target database to see the data changes that were made based on the contents of the source database. Building a Scripted Activity This section describes the steps in creating a Scripted Activity. It describes how to set up and run an activity that will execute an LSX agent. An agent can consist of one or more simple actions or formulas to manipulate data in your databases, custom LotusScript commands to perform a number of different functions, or Java statements, each of these possibilities providing greater functionality to your LEI Activities. Note If you have installed LEI but not yet migrated your LEI Release 3.n scripts from LEI LSX format to LC LSX format, you should perform that migration now, prior to attempting to use those past-version scripts. The LEI LSX format has been retired and LEI LSX format scripts are no longer useable in LEI. 338 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Example 1 This example demonstrates how to access, edit and use two scripts (agents) that are supplied and installed in the LEI Script Vault (leivlt6.nsf). Both of these scripts will access the sample package tracking database (leipackagetrack.nsf) supplied with LEI. The first script will perform a direct transfer from the DATA form in the leipackagetrack.nsf and create and insert this data into a new form or table (depending on the target). The second script will perform a keyed replication between the two tables from the Direct Transfer Activity. There are two ways to access the supplied agents where the scripts are written. • Open the LEI Script Vault leivlt.nsf from the Notes desktop. Right-click and select Open (database) from the resulting menu. Select your server and then select the LEI Script Vault (leivlt6.nsf). The icon will be placed on your desktop. Right-click on it and select “Open in Designer.” Choose File - Database - Open from the Notes top bar. Specify the server name and .nsf file name (or database icon) from the resulting menu and then click Open. Once the database is open, choose View Design to open Domino Designer. Select Agent from the Domino Designer Navigator. • Start Domino Designer and then open the LEI script vault. Choose File - Database - Open from the Domino Designer top bar. Specify the server name and .nsf file name (or database icon) from the resulting menu and then click Open. Expand the Shared Code section and select Agent from the Domino Designer Navigator. To see the code for a specific agent, double-click on the agent name in Domino Designer. The code is categorized under the terms Options, Declarations, Initialize, and Terminate. Appendix C: Tutorials 339 Step 1: Create the Connection Documents In this step, you will create Connection Documents to the source package tracking database (leipackagetrack.nsf) and to the target database, which in this case is the sample employee database (leiempsamp.nsf). Creating the Source Connection 1. From the LEI Administrator, click Add Connection - Connection to Notes. 2. In the Name field, enter Packagetracking. 3. In the Notes Server field, optionally enter the name of your server or leave blank if your server is local. 4. Specify the Notes Database file name leipackagetrack.nsf. 5. Leave all other options set to their default values. 6. Save and close the document. Creating the Target Connection 1. From the LEI Administrator, click Activities - Add Connection Connection to Notes - OK. 2. In the Name field, enter myNotes. 3. In the Notes Server field, optionally enter the name of your server or leave blank if your server is local. 4. Specify the Notes Database file name leiempsamp.nsf. 5. Save and close the document. Step 2: Create the Direct Transfer Activity 1. Open the Package Tracking database by selecting File - Database - Open from the Notes top bar. Specify the server name and leipackagetrack.nsf file name from the resulting menu and then click Open. 340 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 2. Click on the “All documents” view. You will use information from the “Data” key. Appendix C: Tutorials 341 3. Open the LEI Script Vault (leivlt6.nsf) in Domino Designer. 4. Double-click the “PackageTrack Direct Transfer” agent. Note The code for a specific agent is categorized under the terms Options, Declarations, Initialize, and Terminate. The Declarations section contains all the pertinent information for the agent to run correctly (for example, this contains the name of the Source connection document). The script itself is written in the Initialize section. Comments are added where appropriate. 5. At this point, you will create the Scripted Activity to call your LSX agent. In the LEI Administrator, select Add Activity - Scripted. 6. In the Name field, enter Direct Transfer. 7. In the Agent Server field, enter the name of your server. Leave this field blank if your server is local. 8. In the Agent Database field, enter leivlt6.nsf (LEI Script Vault database). 9. For the Agent Name, enter Package Tracking Direct Transfer. 10. In the Category field, enter Samples. 342 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 11. Leave all other options set to their default values. 12. Save and close the document. 13. Start the Activity. Appendix C: Tutorials 343 14. Check the LEI log and you will see that eight records have been fetched from all connections (source), and eight records have been inserted into all connections (target). Step 3: Perform Primary Key Replication In this script, you will perform a replication from the leipackagetrack.nsf database to the leiempsamp.nsf database. The script being used in this step is called “PackageTrack Primary Key Replication,” and can be viewed in Domino Designer. 1. Open the leipackagetrack.nsf database in your Notes client. 2. Choose Create and then Package. The following form appears. 3. Add your own information to the Package Tracking Data entry form. 4. Save and close the document. 5. Create a Scripted Activity to call this script by selecting Add Activity Scripted from the LEI Administrator. 6. In the Name field, enter Replication. 7. In the Agent Server field, enter the name of your server. Leave this field blank if your server is local. 344 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 8. In the Agent Database field, enter leivlt6.nsf. 9. In the Agent Name field, enter Package Tracking Primary Key Replication. 10. In the Category field, enter Samples. 11. Leave all other options set to their default values. 12. Save and close the document. 13. Start the Activity. Appendix C: Tutorials 345 14. Check the LEI log. You will see that seventeen records have been fetched from all connections (source and target), and one record has been inserted into all connections (target). Example 2 This example demonstrates how to perform a simple action on data in the sample database leiempsamp.nsf using a Scripted Activity. Step 1: Create the Agent There are two methods to create an agent in the database leiempsamp.nsf. From within leiempsamp.nsf you can choose Create - Agent from the Notes menu or you can choose the Create Agent button when creating the Scripted Activity Document. For this example, you will create the agent when you create the Activity. 1. Use the LEI Administrator to create a new Scripted Activity. 2. In the Name field, enter Scripted ActivityA. 3. Click the Create Agent button at the top of the document. 4. A dialog box will prompt you for a server name if the default server name has been removed. Enter the server name where the database leiempsamp.nsf is located. 5. A dialog box will then prompt you for a database name in which to create the agent. Enter leiempsamp.nsf. 6. An Untitled agent window opens. 346 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 7. In the Name field, enter Job Script. 8. In the Runtime Target field, specify all documents in database. 9. Close the property box. 10. Click the Add Action button at the bottom of the window. An Add Action box will appear. 11. In the Action field, specify Modify Field. 12. In the Field field, specify JOB. 13. In the Value field, enter ENGINEER. 14. Select Replace value. This action will replace the value in the JOB field for each document in the leiempsamp.nsf database. 15. Click OK. 16. Save and close the agent. The leiempsamp.nsf database remains open. Appendix C: Tutorials 347 Step 2: Complete the Scripted Activity Document 1. Press Escape to return to the Scripted Activity that you created. 2. Click the button in the Agent Name field and choose Job Script from the list of agents. Note If you want to have a specific LEI server execute the activity, specify the name of that server in the Designated Server field under the Activity Execution Options tab. 3. Enter the Category Samples. 4. Leave all other options set to their default values. 5. Save and close the document. Step 3: Start the Activity To run the activity and view it, complete the following steps. 1. Display the Samples category of Activities in the LEI Administrator. 2. Select the Scripted Activity that you just created (named Scripted ActivityA). 3. Click Start from the LEI Administrator. 4. You should see Scripted ActivityA begin to execute. Step 4: View the Log and Activity Results 1. When the Activity is complete, select its name from the LEI Administrator and click Log. The activity should be logged as “Started” and “Finished.” 2. Open the leiempsamp.nsf database and notice the Job field. Each instance has been replaced with the value “ENGINEER.” 348 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Building an Advanced RealTime Virtual Fields Activity The most common use for Advanced RealTime Virtual Fields activity is to give access to external system databases such as DB2, Oracle, or Sybase, to Notes users. This section provides an example of how to give Notes users information about employees, where the employee information is stored in an external system database and might look something like the leiempsamp.nsf sample database supplied with LEI. The EMPNO key field and the ENAME, HIREDATE, SAL, JOB, and DEPTNO data fields are used. You can perform the steps in this example yourself by transferring this Notes data to an external system database, if you have not already done so, using an LEI Direct Transfer Activity. The process for creating an LEI Direct Transfer Activity was described in a previous section, “Building a Direct Transfer Activity.” Note To operate the Advanced RealTime Activities, the LEI server must be installed on the same Domino server as the Notes database that Notes end-users will access for Advanced RealTime lookups. Follow these steps to create a simple Advanced RealTime Virtual Fields application. Step 1. Gather the Necessary Information Identify the table name, description (metadata) and necessary user name and password in the external system database. If you followed the previous Direct Transfer example, the table name is Employees, the field names are as above, and, of course, you know the user name and password needed for the external system database. Step 2. Create the Form in the Notes Database Create a form in the Notes database having corresponding fields for the key(s) and data fields. The field names do not need to match those used in the external system database, however the datatypes should match. Enter a form name of your choice. Create a view from which you will later select documents. For this example, use the view name “Employee Lookup.” Step 3. Create the Connection Documents Create a Connection document to the external system database using the information gathered in step 1 above, or reuse one that you created for a previous example, Sample 1 Destination. Appendix C: Tutorials 349 Step 4. Create the Virtual Fields Activity Create the Virtual Fields activity in the LEI Administrator using the above Connectors. Map the key and data fields that correspond. This Virtual Fields activity example uses the “Document Open” Monitor, which causes a lookup against the DB2 table EMPLOYEE when end-users open the form EMPS with only the keyfield EMPNO populated. Step 5. Initialize the Keys in the Notes Database Click the “Initialize Keys” button in the Virtual Fields activity document. The Notes database above must be initialized by creating key documents, which contain the keys that refer to the records in the external system database. Do this by initializing keys. 350 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 6. Start the Virtual Fields Activity 1. Return to the LEI Administrator database and use the Activities view to locate your activity. 2. Either choose Start or open the Activity and establish a schedule for when the Virtual Fields activity will be operational for Notes end-users. 3. Enable the schedule, and save the document. Note The Virtual Fields activity will continue to run until you either stop it by choosing Stop or the schedule expires. 4. Refresh the Activities view until the Activity status icon becomes a beacon. This icon indicates that the activity is now running. Step 7. Use the Virtual Fields Activity Use the application by opening the monitored Notes database and opening one of the documents containing a key value. The first open will take longer because the connection has to first be established with the external system data base. The data fields are then populated by the Virtual Fields activity from the external system database. This completes the example of using a Virtual Fields Advanced RealTime activity. Building a Dynamic Advanced RealTime Virtual Fields Query for Notes or Web Clients This section provides information about using your Notes client or Web browser to invoke a Virtual Fields activity that accepts a key value input by the user for querying an external data source. This example shows how to extend the functionality of the Advanced RealTime activity to provide a dynamic query capability. Example Contents and Setup Included with your LEI installation disk is a sample database leipackagetrack.nsf. This database provides the Lookup and Status forms used in the example. The database also includes sample raw data that may be used to populate your external system database for use with the example. Appendix C: Tutorials 351 Overview There are several pieces needed to create Advanced RealTime Virtual Fields Dynamic Notes or Web based queries: • An external system database containing the information that you want to present to users in Notes • A Notes database on your Domino server, such as leipackagetrack.nsf • A Virtual Fields (or Virtual Documents) activity which resides on the same Domino server and connects the two To follow this example you will move the sample data to the external system database, create two forms and a view in your Notes database, and then you will create a Virtual Fields activity to monitor the Notes database. This example database has the necessary forms and views if you do not wish to create them yourself. Step 1. Setup Copy the leipackagetrack.nsf database to a directory on the Domino server where LEI is installed. Note To modify the forms, open leipackagetrack.nsf in Domino Designer. Step 2. Create or Choose an External System Database to Monitor Within the leipackagetrack.nsf database there is data on packages that can be transferred to your external system database. These documents or records have been created with the Package form and can be viewed with the Packages view. You can use LEI to move this data to your external system database, or choose another similar table of your own, or create the table. The key field named “CODE” is what is important here. If you are using a table of your own, identify the key field you want to use for lookups and use it in the appropriate places in the later steps. This example uses PackageID on the Notes side to illustrate that the field names need not be the same in both databases. If you use an LEI Direct Transfer Activity to move this data to Oracle, select the option for “No Long Column” in the Connection document. Otherwise the first text column will be made a LONG in Oracle. 352 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 3. Create the Lookup Form in the Notes Database If you are using leipackagetrack.nsf, you can use the form “Lookup” included in this database. The key field that will be used by LEI for the lookup is what is important here. In this example it is PackageID. You can put information on this form to guide the user. Note There is a view “Query” referred to in the following formula; you will need to create that view after the “Lookup” form is completed. There is a button in the form for “Lookup Package” that contains the following formula: @PostedCommand ([FileSave]); @PostedCommand ([FileCloseWindow]); @PostedCommand ([FileOpenDatabase]; @DbName; "Query"; @Text(PackageID)); @PostedCommand ([OpenDocument]) This code is used when running the application from Notes but is ignored when running the application from the Web. Note The create event should not be monitored when performing dynamic queries. This can lead to an unwanted record being inserted in the external system or the error “duplicate key value specified.” 1. Save the document with the PackageID that the user wants entered into that field. 2. Close the window. Note If you do not close the window, an orphaned “Untitled” document is left. 3. Locate the document in the Query view where the indexed key field PackageID is the same as the one just saved. 4. Open the document. The view selection formula will then display the Status form. Appendix C: Tutorials 353 Description of How this Works From the Web, Domino will treat the button as a POST command and will create a document in the database and then respond to the formula in the $$Return field, which should be hidden. The $$Return field will generate a URL that will open the newly created document in a view called Query just as it happens with the Notes client. In this example, the $$Return field contains the following: @If (PackageID=""; "Please supply a tracking number before pressing the LOCATE button"; "[/" + @Subset (@DbName; -1) + "/Query/" + PackageID + "?OpenDocument]" 354 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 4. Create the Status Form in the Notes Database Create another form to return the information to your user. In the example it is called “Status.” This form must also use the key field “PackageID.” The Virtual Fields activity will be asked to “Monitor any form;” it will respond to the opening of this “Status” form when the original document created by the user is opened from the view, “Query.” By using a Form Filter formula (described in Step 7) only the two forms of interest will trigger the Virtual Fields activity. Step 5. Create the Query View in the Notes Database You will need to create a view to find the document that was just created in the last step. This view must be indexed on the key field, in the example PackageID. The view selection formula is: SELECT (@Contains(Form; "Lookup")) Under the View Properties Advanced tab, put the following formula: @If(@IsNewDoc;"Lookup"; "Status") This will switch the form used to the Status form when opening a document that has been saved using the Lookup form. Appendix C: Tutorials 355 Step 6. Create the Connectors to the Source and External System Databases You will need two LEI Connectors; one to the external system database: And one to the Notes database being monitored by Advanced RealTime: 356 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Step 7. Create the Virtual Fields Activity Create the Virtual Fields activity in the LEI Administrator that will monitor the Notes database for Open events. This activity will use the Connection document created in step 3 of the previous example, “Building an Advanced RealTime Virtual Fields Activity,” to link the external system database to the Notes document using the key field you have chosen. In this case the key field in Notes is PackageID and in the external system database is CODE. Choose the form Status so that the data fields are available for you to map. You need to monitor the Lookup form in order to catch its Open event, so you must check “Monitor All Forms.” However, in the sample database there are other forms that are unrelated to the functioning of this application, exclude them by using a filter formula “FORM = “Status” ! “Lookup.” Appendix C: Tutorials 357 Step 8. Perform the Final Steps 1. Ensure that you have populated your external system source with the leipackagetrack.nsf sample data as mentioned earlier. You can use a Direct Transfer activity to do this. 2. Enable (schedule) the Advanced RealTime activity you have created if you want to make it available for a specific interval of time, or click the Start! button on the Activity Document to make it active until you close it manually. 3. Verify that the Activity is active by refreshing the view and watching for the Active icon. If the Error icon appears, you must go back and check your work. 4. From a Notes client create a Lookup document from the Actions menu and type in a tracking number in the PackageID field. Refer to the leipackagetrack.nsf database for example numbers. 5. Click the Locate button to save the Lookup document in the database. The same document is reopened in the view. The selection formula in the view then switches the form to Status when it sees that the document has been previously saved. 358 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 6. From a Web browser access the Lookup form using a URL similar to this: http://servername/leipackagetrack.nsf/Lookup?OpenForm Note You will need to modify this to reflect the name of your own Domino server and any other changes you might have made in the names used in your application. 7. Type in a tracking number (PackageID field). Refer to the leipackagetrack.nsf database for example numbers. 8. Click the Locate button. When you click the Locate button, the Lookup document is saved in the database and the $$Return text is passed to the Domino HTTP process. The new document is reopened. The Advanced RealTime activity is activated, takes the value submitted to the Lookup form, and maps it to the Advanced RealTime activity definitions and Status form. The PackageID value is then sent to the external system source and the table records are searched according to the PackageID value. Results of the search are sent back to the Web application as directed by the URL specified in the Lookup form. Additional Advanced RealTime Examples This section provides information about: • Building a Virtual Fields or Virtual Documents activity that uses Virtual Attachments • Using filter formulas, storage options, monitor orders, and stored procedures in Virtual Fields activities Building a Notes Application for a Virtual Fields or a Virtual Documents Activity The Notes application being monitored by a Virtual Fields or Virtual Documents Activity must include data fields that map to data fields in the external data source. In addition, the Virtual Fields Activity must include at least one field that maps to a key field in the external data source. A key field is one or more fields that are used to uniquely identify the data. A Virtual Fields Activity can include multiple key fields. The next section gives an example of an application that could be used with a Virtual Fields Activity or Virtual Documents Activity. Appendix C: Tutorials 359 Example Application Shown below is a Notes application that monitors employee information in an external data source. To see what the design view of this application looks like, see the following section. 360 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Example of the Application Design View The illustration below shows the design view of the example application. In this application there are labels next to the data fields. These data fields map to the data fields in the external data source this application will monitor. The Virtual Fields Activity or Virtual Documents Activity that uses this application also defines the field mapping in the mapping area of the activity form. Appendix C: Tutorials 361 Virtual Fields Activity Example Shown below is an example of a Virtual Fields Activity that uses the example application. The field mapping in the Mapping section of the document maps to the example application above. Note You must initialize fields for the Virtual Fields Activity to populate the Notes application you developed. Initialize fields by selecting “Initialize Keys” from within the Virtual Fields activity by opening the activity and clicking the “Initialize Keys” button. Caution Initialize keys only one time. Initializing keys more than once results in duplicates. 362 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Documents Activity Example For Virtual Documents to work, you must add four additional fields to the external system database. These additional fields must be mapped to Notes ID information fields. See “Required External System Fields for Virtual Documents Activities” in the Virtual Documents Activities chapter for complete information. Note You can use the Create Table Utility action button to create these fields for you, however, see the Virtual Documents Activities chapter for complete details. Below is an example of a Virtual Documents Activity that makes use of the example application developed earlier in this chapter. The field mapping in the Mapping section of the document maps to the fields in the example application. Note In Virtual Documents there is no need to initialize keys. Initialization occurs automatically. Appendix C: Tutorials 363 Using Virtual Attachments The Virtual Attachments feature allows you to use Notes to add attachments to the external system database. Once added, they can be accessed through the virtual views. To use the Virtual Attachments feature, you must create a table and enter fields in the external system that accommodate Virtual Attachments. The fields that you create at the external system must be in a table expressly created to hold these fields and these fields only. Note You can use the Table Creation Utility button to automate the creation of an attachment table. See “Using the Create Virtual Attachment Table Option” in either the Virtual Fields Activity or Virtual Documents Activity chapters for complete information. The required Virtual Attachment field names are as follows: • EIDBID • EIFILEID • EIFILESIZE • EINOTEID • EIFILENAME • EICONTENTS Note The data types required for each of these fields vary according to the external system. See the following table: Attachment Table Fields External System Data Types Oracle Informix Sybase (a) DB2 (b) Varchar2(16) or char(16) char(16) char(16) character(16) char(32) EIDBID** number int int integer int EIFILESIZE (c) number(8) int int integer int EINOTEID (d) number(10) int int integer int EIFILENAME varchar2(255) varchar(255) varchar(255) varchar(255) varchar(255) EICONTENTS long raw image blob(1048576) image byte SQL Server ** Index these fields at the external system to increase performance. a) In Sybase, column names must be in upper case, for example, EINOTEID. b) DB2 — with up to a 1MB attachment. c) EIFILESIZE column length may be adjusted, but it must remain an integer (X,0). 364 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide d) EINOTEID is a DWORD, or an unsigned 32-bit integer. According to the formula 2^n - 1, this gives a maximum EINOTEID of 4,294,967,295 which requires 10 digits. You identify the table where these fields reside by entering the name of the table in the “Attachment table” field in the activity form. In the illustration below the table that is to be used for attachments is called MY_ATTACH. Note Any attachments that were not entered into the external system through the Virtual Attachment process will not be accessible through either Virtual Fields or Virtual Documents. That is, if you added attachments to the external system database through another process, for example, SQL commands, you cannot access then through Virtual Fields or Virtual Documents. Of course, you can create the table using SQL commands. Appendix C: Tutorials 365 Using Virtual Fields Options This section provides examples of using filter formulas, storage options, monitor orders, and stored procedures in Virtual Fields activities. Using Filter Formulas This section provides examples of how you would use the Filter Formula option in conjunction with the Monitor Order option to achieve specific results. Saving Data to an External Data System Scenario: You want to store data from a Notes document in an external data system, such as a DB2 or Oracle database, when the document data becomes stable. When each document is stable and ready to store on the external system, you would change its status from “under review” to “publish.” How to Configure: In this case, you would set up a Virtual Fields Activity that monitors the Notes application. In the Virtual Fields Activity, you would use a filter formula that includes only documents that have been marked “Published.” At that point the information from the Notes document will be written to the DB2 database, where it would be available to other applications. Any documents that are still “under review” will not be stored in DB2. Accessing Different External Sources using the Same Notes Application Scenario 1: You want to use a single Notes front end to interact with information that is potentially in two different tables. For example, the North East sales information is in a different table than the South West region. In this case, you would set up one Virtual Fields Activity for each region with different external data sources, but using one Notes document as the front end. You could use a filter formula that would detect the region of information/account request and go to the appropriate external source data. Scenario 2: Suppose that you have two product catalogs in different databases, but want to present the data using the same front end Notes application. You want to use a single Notes front end, but dependent on a field, you want to control where the data is coming from. 366 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Set up identical Virtual Fields Activities monitoring a single Notes application, but have each Virtual Fields Activity use a filter formula to handle part of the request. Using Data Storage Options This section provides some examples of how and when you would use the Data Storage options in Virtual Fields Activities. Building Views Any data that you want to use when building views must reside in the Notes document. Use the option “Leave Selected real-time Fields in Document.” When you select this option, you can choose the fields that you want to reference in a view from the list of available fields. Storing a Static Copy of External Data If you want to store a static copy of the external data, select “Leave All realtime Fields in Document.” Note that if the external source data changes, the data in the document is not updated until the document is reopened. Using the Monitor Orders Option In this scenario, the external data for your Notes Human Resources Information system is in multiple tables. A “join” of the data is required to tie these various tables together to populate the single Employee Information form in the Notes application. In some cases, a join from the database may not be possible, because the tables may reside in different databases (one in Oracle, one in Sybase, for example). Information in the Employee Information form includes Employee Name, Department Name and Location Information. The following table explains which fields are used from which table and how they will relate to the Notes form. Fields Used: Employee Information Form EMPLOYEE Table EmpID EmployeeID EmpName EmployeeName DepartmentNo (may be hidden) DeptNo Department DEPARTMENT Table (access with monitor order 2) LOCATION Table (access with monitor order 3) DeptNo DeptName continued Appendix C: Tutorials 367 Employee Information Form EMPLOYEE Table DepartmentLoc (may be hidden) DEPARTMENT Table (access with monitor order 2) LOCATION Table (access with monitor order 3) DeptLoc LocCode Location LocName … To do this join, you will create three Virtual Fields Activities that each monitor separate tables: one to monitor the Employee table, one to monitor the Department Table and one to monitor the Location Table. The first Activity (using a Monitor Order of 1) will provide data that will be used by the other Activities (with Monitor Orders of 2 and 3) as “keys” to their tables. The Notes Form “Employee Information” will need to contain those fields that will act as “keys” to the secondary tables. These fields may be “hidden” from the user. If you expect updates to be made through the Notes application to these secondary tables, select the Option “Leave selected real-time fields in Documents” and list all the fields which are used as keys to the external source (DeptmentNo, DepartmentLoc). Note Activities with monitor orders of 2, 3, 4, and so on, must monitor an Open event. They may monitor other events too, but one of the events must be an Open event. Depending on the nature of the activity, the activity with monitor order 1 may be required to monitor an Open event as well. 1. Create a connection document for each of the external data sources: one for the employee table, one for the department table, one for the location table. 2. Create a Virtual Fields Activity based on the Employee Information Form and the Employee Table. Select all fields that will be used by the form and/or used as a key to another field. In this case, select at least the following: EmployeeName, DepartmentNo. Notes Employee Table EmpID as the key EmployeeID as the key DepartmentNo DeptNo 368 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide 3. Create a second Virtual Fields Activity based on the Employee Information Form and the Department Table. Select the following fields: Notes Department Table DepartmentNo as the key Deptno as the key Department DeptName Set the Monitor Order for this activity to 2. This will allow the first activity to retrieve the information from the employee table before looking into the department table for the DeptName field. 4. The third activity will work like the second, except it monitors the Location Table. It will use a monitor order of 3 because its lookup depends on information provided by the second Activity. Notes Location Table DepartmentLoc as the key LocCode as the key Location LocName Using Stored Procedures in Sybase/SQL This section provides an example of a Virtual Fields Activity that uses stored procedures with a Sybase external data source or a SQL (using Sybase) external data source. In this example there are four stored procedures for the table addrbook in Sybase. When using these stored procedures, the key fields used must be FirstName then LastName and the mapped fields must be (MailDomain, MailServer, MailAddress, CompanyName, and State). The stored procedures, table name and fields must use the correct case since Sybase is case sensitive. To use a stored procedure, enter the name of the stored procedure in the options section for the appropriate event in the Virtual Fields Activity document. In this example, these would be: Create: QEInsertaddrbook Open: QESelectaddrbook Update: QEUpdateaddrbook Delete: QEDeleteaddrbook Appendix C: Tutorials 369 Below is an illustration of a Virtual Fields Activity document showing how these stored procedures are entered in the Stored Procedure fields of the corresponding document event options. Following the illustration is an example of how you can test the stored procedures directly in SQL through the database client. 370 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Testing the Stored Procedures in SQL The examples below show how you would execute the stored procedures in SQL through the database client, in this case Sybase. You can test that the stored procedures work properly by executing them directly as shown below. Note This example assumes that the table “addrbook” exists with the columns shown. execute QEInsertaddrbook ’OurDomain’, ’ThisServer’, ’OurAddress’, ’Open Research’, ’NH’, ‘Adelino’, ’Fontes’ execute QESelectaddrbook ‘Adelino’, ’Fontes’ execute QEUpdateaddrbook ’OurDomain’, ’ThisServer’, ’OurAddress’, ’Open Research’, ’NH’, ‘Adelino’, ’Fontes’ execute QEDeleteaddrbook ‘Adelino’,’Fontes’ Stored Procedure Definitions Below are the actual stored procedure definitions used in the preceding example. Note that these are Sybase procedures; stored procedures for other data sources may be different. create procedure dbo.QEInsertaddrbook( @FirstName varchar(20), @LastName varchar(20), @MailDomain varchar(20), @MailServer varchar(20), @MailAddress varchar(20), @CompanyName varchar(20), @State varchar(20) ) as INSERT INTO addrbook (FirstName,LastName, MailDomain, MailServer, MailAddress, CompanyName,State) VALUES (@FirstName, @LastName, Appendix C: Tutorials 371 @MailDomain, @MailServer, @MailAddress, @CompanyName, @State) create procedure dbo.QESelectaddrbook( @FirstName varchar(20), @LastName varchar(20) ) as select FirstName,LastName, MailDomain, MailServer, MailAddress, CompanyName,State from addrbook WHERE FirstName = @FirstName and LastName = @LastName create procedure dbo.QEUpdateaddrbook( @FirstName varchar(20), @LastName varchar(20), @MailDomain varchar(20), @MailServer varchar(20), @MailAddress varchar(20), @CompanyName varchar(20), @State varchar(20) ) as UPDATE addrbook SET MailDomain=@MailDomain, MailServer=@MailServer, MailAddress=@MailAddress, CompanyName=@CompanyName, State=@State WHERE FirstName=@FirstName and LastName=@LastName create procedure dbo.QEDeleteaddrbook( @FirstName varchar(20), 372 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide @LastName varchar(20) ) as DELETE FROM addrbook WHERE FirstName=@FirstName and LastName=@LastName Using a DB2 Stored Procedure In DB2 the parameter types are interpolated from the Notes datatypes. For example, if the Notes key field is a number, the DB2 parameter type will be DOUBLE. Your stored procedure should account for possible differences in input types (or it should at least check the input type). Model your stored procedure on the following example. /* Stored procedure Example */ #include <memory.h> #include <string.h> #include <c:\sqllib\include\sql.h> #include <c:\sqllib\include\sqlda.h> SQL_API_RC SQL_API_FN sf_proc (void *reserved1, void *reserved2, struct sqlda *inout_sqlda, struct sqlca *ca) { /* Declare a local SQLCA */ EXEC SQL INCLUDE SQLCA; /* Declare Host Variables */ EXEC SQL BEGIN DECLARE SECTION; char *select_stmt = "SELECT NUMBER2, TEXT1, TEXT2, MODIFIED, NUMBER1 FROM SIMPLE WHERE NUMBER1 = ?"; long int number1; EXEC SQL END DECLARE SECTION; Appendix C: Tutorials 373 if (inout_sqlda) { if ((inout_sqlda->sqlvar[0].sqltype == SQL_TYP_INTEGER) || (inout_sqlda->sqlvar[0].sqltype == SQL_TYP_NINTEGER)) number1 = *((long int *) (inout_sqlda->sqlvar[0].sqldata)); else if ((inout_sqlda->sqlvar[0].sqltype == SQL_TYP_FLOAT) || (inout_sqlda->sqlvar[0].sqltype == SQL_TYP_NFLOAT)) number1 = (long int) *((double *) (inout_sqlda->sqlvar[0].sqldata)); else number1 = 0; EXEC SQL WHENEVER SQLERROR GOTO ext; EXEC SQL PREPARE stmt FROM :select_stmt; exec sql DECLARE curs CURSOR FOR stmt; exec sql OPEN curs using :number1; } ext: memcpy(ca, &sqlca, sizeof(struct sqlca)); if (inout_sqlda) *(inout_sqlda->sqlvar[0].sqlind) = -128; return(SQLZ_DISCONNECT_PROC); } 374 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Using an Oracle Stored Procedure An Oracle stored procedure can be used in a Virtual Fields Activity or with an LC LSX script. However, the stored procedure being called must have its parameters configured correctly for the call. When you browse for a stored procedure in the Connection Document, a dialog box appears, listing all the stored procedures in the external system database. When you choose a stored procedure, the parameters of the chosen stored procedure are automatically listed in the Connection Document. If you do not specify a stored procedure in the Connection Document, you can select a stored procedure and its associated parameters from the Virtual Fields Activity Document. In either case, the stored procedure is entered as the default stored procedure in the Activity events in the stored procedure field, followed by its list of parameters. In most cases, returned values are handled by the external system database. In the case of Oracle, the Lotus Connector for Oracle 7 or 8 handles returned information. This is transparent to you, however the person responsible for writing Oracle stored procedures should be aware of it. Input Values and Named Parameters Input values are provided to Oracle procedures as named parameters. This requires that the parameters in Oracle use the same names as the fields being provided as input values. The inputs being provided include key values when they are used in the context of a keyed operation (selection, update, or delete context), and data values when relevant (insert or delete context). The input value data types provided by the Lotus connection are selected as the closest match to the LEI data type and are converted by Oracle to the procedure parameter-defined data types, provided that the conversion is supported by Oracle. Using Output Parameters to Return Results Oracle differs from other RDBMS Connectors in that there is no way to return a result set from a procedure. Therefore, the Lotus Connector for Oracle 7 or 8 supports output parameters as a way of returning results from a stored procedure. This requires additional information to be available at the time the procedure is called, specifically the context of the call and the output parameter names. This information will be automatically provided by Virtual Fields activities, but must be manually specified when calling Oracle procedures from an LC LSX script. The context indicates whether the procedure should expect and specify output parameters, and the parameter names are provided as a property of the procedure call request to the Oracle connection. The output parameters must be standard data types — row sets may not be returned. This restricts the result set from an Oracle procedure to a single row. Any parameters that are indicated as input Appendix C: Tutorials 375 parameters and are also in the output parameter list will be provided as input/output parameters. Example of an Oracle Stored Procedure The following is an example of an Oracle stored procedure body. This is the format that would be required for the Open event of a Virtual Fields Notes activity assuming that the key field is called NUMBER1 and the data fields are called NUMBER2, TEXT1, and TEXT2. In this context, the key field is the input parameter, and the result set is expected to include the data fields and the key field. To accommodate the fact that one of the output parameter names is the same as a key value in the select statement, the parameter keys should be copied to local variables to avoid scoping problems in the procedure. (NUMBER2 out tablename.number2%TYPE, TEXT1 out tablename.text1%TYPE, TEXT2 out tablename.text2%TYPE, NUMBER1 in out tablename.number1%TYPE) IS BEGIN DECLARE number1_request tablename.number1%TYPE := number1; BEGIN SELECT t.NUMBER2, t.TEXT1, t.TEXT2, t.NUMBER1 INTO number2, text1, text2, number1 FROM tablename t WHERE t.NUMBER1 = number1_request; END; END; 376 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Appendix D Using Virtual Fields with Virtual Documents This appendix provides information about using Virtual Fields and Virtual Documents activities. It is assumed that you are familiar with these two activity types. Introduction This appendix builds on content introduced earlier in this guide in the Virtual Documents and Virtual Fields chapters. See those two chapters for more detail on these two related activities. A Virtual Document is essentially equivalent to a native Notes document in every way, except that it is not stored in an NSF and all its data is external to the NSF, as a result you can add Virtual Fields to the Virtual Document. You do this by simply creating a Virtual Fields activity that monitors the same form that a Virtual Documents activity monitors. This effectively adds virtual fields to a document that is itself already virtual. The concept may seem odd at first, but the ability to layer virtual fields over a virtual document adds significant extensibility and functionality not available to Virtual Documents alone. It also adds a level of complexity, with potential pitfalls, that needs to be considered before using this functionality in this manner. The user must be aware of his or her needs and whether this approach is suitable to solve a particular problem, or whether it can be solved more simply by using only Virtual Fields or only Virtual Documents. This section is intended to help you answer this question and it describes how to properly implement this advanced solution. 377 Advantages to Using Virtual Fields with Virtual Documents There are two potential advantages to adding Virtual Fields to a Virtual Document. • The ability to include data from other external data sources, potentially from entirely different external data source systems, into a Virtual Document. • The ability to utilize some of the functionality from Virtual Fields activities, which is inherently not available in Virtual Documents activities, within in a Virtual Document. It very important to understand that a Virtual Document stands on its own, and that, by definition, all its fields are virtual. These are the fields which are mapped in the Virtual Documents activity. A Virtual Documents activity monitors a single Notes Form, and all the mapped fields in that form are said to be virtualized and exist in a single table. Consequently, each table row corresponds to single Virtual Document. In addition, all the other various elements which comprise a Notes document are also virtualized in a external data source table, and hence the entire document is said to be virtual. In essence, a Virtual Documents activity instantiates a complete Notes document, all of whose components and data exist external to the Domino NSF. This is the fundamental difference between Virtual Documents activities and Virtual Fields activities. While Virtual Documents virtualizes the entire document, Virtual Fields virtualizes individual fields and utilizes key documents stored in the NSF to hold the document level information and the all important key fields used to map a document to a particular external data source table row. Virtual Documents removes the need to have and maintain stub documents, but at the expense of not being able to map various fields within a single document to different external data sources, nor can you have some fields stored natively (not virtual). Said another way, the document level nature of Virtual Documents means that all the fields in a Virtual Document always map to a single external data table row which describes the document as a whole; by contrast, multiple Virtual Fields activities can supply external data from multiple external sources to various virtual fields within a single document. The result of using a Virtual Fields activity to monitor the same form as a Virtual Documents activity is to add virtual fields to the virtual fields that are inherently part of the virtual document itself. If Virtual Field activity maps its fields from some other supported 378 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide external system, you’ve essentially added the ability to create a Virtual Document that accesses data from multiple external systems, which is something you cannot do with Virtual Documents alone. What Happens to the Virtual Field Key Documents You may wonder how you can use Virtual Fields if you haven’t created any key documents to supply keys. The short answer is that you do have key documents. The key documents are now actually the Virtual Documents themselves. This will be discussed in more detail later, but the primary requirement is that one or more fields in the Virtual Document (mapped by the Virtual Documents activity) can be used as key(s) for the Virtual Fields activity. It is not necessarily advantageous to think of Virtual Documents as strictly a substitute for Virtual Fields key documents, or as a way to use Virtual Fields without the need for maintaining and synchronizing native stub documents. The fact that the key documents are now virtual does not alleviate these issues, and in some ways adds more overhead. Think of Virtual Fields as a way to extend the external data access capabilities of Virtual Documents to include data from multiple external sources and to add some extra functionality. Thought of correctly, Virtual Fields adds power to Virtual Documents, rather than Virtual Documents makes Virtual Fields easier to use. Appendix D: Using Virtual Fields with Virtual Documents 379 Additional Functionality Some Virtual Field functionality can be incorporated into Virtual Documents by configuring one or more Virtual Fields activities to monitor the same form as a Virtual Documents activity. Most notable of these is the ability to use multivalue data fields within a Virtual Document; fields mapped by the Virtual Documents activity cannot be multivalue fields. Any multivalue fields in a form monitored by a Virtual Documents activity would instead be mapped by a Virtual Fields activity with the appropriate multivalue parameters set. When viewing a Virtual Document, the fields mapped with the Virtual Documents activity will appear together with the multivalue fields, and any other fields, mapped by the Virtual Fields activity. Virtual Fields also has the notion of monitor order, whereby multiple Virtual Fields activities monitoring the same form can be set up to “run” in a specific order. In a typical case, the first Virtual Field activity using a key(s) supplied from the “key document”, populates a set of virtual fields in the document. The populated virtual fields from the first activity then supplies the key(s) for the next Virtual Field activity in the monitor order and so on. In some circumstances this ability to sequence activities can be very useful, but it is not possible with Virtual Documents alone since a single Virtual Documents activity exists in a given form. By adding two or more Virtual Fields activities to Virtual Documents, the fields mapped by the first Virtual Fields monitor can supply the key(s) for the second activity and so on as before. The only difference is that the Virtual Document supplies the key(s) for the first Virtual Fields monitor. In the context of monitor order processing, you can think of the Virtual Documents activity as always having a monitor order of 0; that is, it always runs first. Adding Virtual Fields to Virtual Documents Adding virtual fields to a virtual document is somewhat of a misnomer since the fields mapped by the Virtual Document activity are of course already virtual. What we are really doing here is adding more virtual fields mapped in one or more Virtual Fields activities. This is simply accomplished by creating a Virtual Fields activity that monitors the same Notes form which is being monitored by a Virtual Documents activity. It’s assumed that you are already familiar with the creation of Virtual Documents and Virtual Fields activities. 380 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide The most important aspect of creating the Virtual Fields activity involves the selection of the key field(s). Just as ‘regular’ Virtual Fields activities require that the key fields exist in the key documents, the key field(s) here must be also be mapped by the Virtual Documents activity. Because the key(s) are mapped by both activities they must be common to both external system tables. Since Virtual Documents activities do not have a notion of key(s), they will appear as any other data field in the Virtual Document mapping section, but they will then be mapped as keys in the Virtual Fields activity. (see figure). In the example [shown in the above figure], empno is mapped as a data field in the Virtual Documents activity along with some other employee information. All these fields map to the external system employee information table. The Virtual Fields activity maps the same empno field to another external system table, along with some other data relating to the employee’s department. The empno field is the only field that need be in common between the two tables and the corresponding activity mapping. When these activities are running, the Virtual Documents activity will first construct the Virtual Document which will essentially provide the needed key(s) for the Virtual Fields activity to be able to access its external system table. The end result is a Notes Document which is a composite virtual Appendix D: Using Virtual Fields with Virtual Documents 381 document; that is, it contains virtual fields supplied by both the Virtual Documents activity and the Virtual Fields activity using data from two different external system tables. This example can be extended further to include one or more additional Virtual Fields activities. These activities could use the same empno field as the key, or use other fields as keys, possibly supplied by other Virtual Fields activities and using the Virtual Fields monitor order capability. Refer to the Virtual Fields activity chapter for more information about monitor order. With some caveats, almost anything that can be done with regular Virtual Fields activities using key documents can be done with Virtual Fields and Virtual Documents together. The next sections will cover some of the special issues which arise when using these two types of activity together. Managing the Key Field Under Different Scenarios As implied earlier, all the Virtual Fields activity key(s) which exist in the Virtual Documents table must also exist in the Virtual Fields table, or else accessing the virtual document will fail because the lookup on the Virtual Fields table will fail. In other words, attempting to open a virtual document that contains a key value that does not exist in the Virtual Fields table results in an Notes open failure. Under normal circumstances this should not happen, but certain improper configurations and/or error conditions could lead to this situation. 382 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Scenario 1: One-to-Many Record Correspondence This situation arises when there is not a strict one-to-one correspondence between the records in the Virtual Documents table and the Virtual Fields table. This could be described as a one-to-many record correspondence. For example, suppose an external system table contains employee data for 100 employees. This table can easily be virtualized into a Domino environment through a Virtual Documents activity in the usual manner. One of the columns in this table provides a department code indicating the department to which employee is currently assigned. Now suppose a second external table contains department data for each of the company’s 12 departments, using the department code as a unique key. Each department has one or more employees assigned to it; this is the one-to-many relationship. The goal is to consolidate employee information with information about the employee’s current department on a single Notes form. This would make the consolidated document appear as a single Notes document which can be manipulated in exactly the same ways a normal Notes document can be manipulated in a Domino environment. As described earlier, the only way to supply these additional virtual fields (department data) to your virtual document (employee data) is by using a Virtual Fields activity which monitors the same Notes form as the Virtual Documents activity, and mapping the department code field as the key in the Virtual Fields activity. However, if the Virtual Fields activity is monitoring ALL Notes events, including creates, updates and deletes, a problem will arise when an employee is terminated, or transferred to another department, or a new employee is hired. Consider the following cases: If an employee Fred Waters in department 4 is terminated, and the Notes application is intended to only contain information about active employees, simply deleting Fred’s document has the following consequences: 1. Fred’s record in the Virtual Documents table will be marked as deleted; this allows Domino to replicate the document deletion to other potential database replicas. 2. Since the Virtual Fields activity is monitoring delete events, it will delete Fred’s former department record from the Virtual Fields department data table, using 4 as the key. 3. All of Fred’s former co-worker’s records in department 4 will no longer be accessible through Notes, since opening the virtual document will result in a Virtual Fields lookup failure for their department data. Appendix D: Using Virtual Fields with Virtual Documents 383 If Fred is transferred from department 4 to department 7, updating Fred’s document has the following consequences, but only if key field updates are allowed in the Virtual Fields activity (they are BLOCKED by default): 1. Fred’s department code in the Virtual Documents employee information table will be updated from 4 to 7 2. Since the Virtual Fields activity is monitoring update events, the department code record in the Virtual Fields department information table will also be updated from 4 to 7, assuming the Virtual Fields activity is set to allow key field updates. 3. Updating the key field with a new value will probably have many negative consequences, ranging from document access problems for other employees still in department 4 to duplicate key errors from the external system table. If Bob Smith is hired as a new employee to help Fred out in department 4, the following will occur: 1. Bob Smith’s record will be inserted into the Virtual Documents table, with 4 as his department code. 2. Since the Virtual Fields activity is monitoring create events, a new row for department 4 will be created in the Virtual Fields department information table. 3. This may lead to duplicate key problems since there is already a department 4, and it really doesn’t make sense since we do not want to reenter department information for department 4, but instead want to add a new employee who will work in department 4. Solution What we really want to do in this scenario is to make the Virtual Fields activity a Read Only activity that simply uses the department information table as a lookup table to provide department information when each employee’s documents is opened. This is very easily accomplished by always setting the Virtual Fields activity to only monitor OPEN events when you have this one-to-many scenario. Set up this way, deletions and updates to an employees document will only affect the employee record in the Virtual Documents employee table. Creating a new employee record will simply involve entering the employee information including the department code. When the new employee’s record is subsequently opened, the correct department information will be displayed along side the employee information. 384 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide A special case arises if you do want to be able to update department information from within an employee’s virtual document. In this situation, allow the Virtual Fields activity to monitor update events, but be sure that the update event option for Key Field Updates is set to Block (which is the default). This will allow you to edit department information, but still allow you to transfer an employee to a different department without running into the problem described above, where the department code key is changed in the department information table as part of the update. If you are already familiar with the ‘regular’ use of Virtual Fields activities, you may recognize this one-to-many scenario. This typically occurs when using multiple Virtual Fields activities with a prescribed monitor order. In this scenario. the first Virtual Fields activity in the monitor order provides the key for second Virtual Fields activity which accesses a read only type lookup table such as the department information table described above. The same problems and solutions described above apply in this case. The only difference in the virtual documents context is that the Virtual Documents activity essentially plays the role of the first activity by providing the key for the Virtual Fields activity. Scenario 2: One-to-One Record Correspondence There may be circumstances where it is desirable to have read/write access to the Virtual Fields table. This would include the case where there is a one-to-one relationship between the records in the two tables. For example, instead of department information in the Virtual Fields table, suppose the table simply included more employee specific information concerning job performance. The key might be the employee’s social security number. In this case, when an employee is terminated and the document deleted from the Notes application, it may be preferable to delete all the employees records from both tables. Also, being able to update information about the employee, including job performance data in the Virtual Fields table, may be desirable. Similarly, a new employee would always result in new records being created in both tables. In this situation you do have a one-to-one relationship between the two tables, as a result, you may not want to block key field updates, because unlike the example where the key was a department code shared by multiple employees, here the key uniquely identifies a single employee. In our example, if an incorrect social security number is changed but the key field update is blocked in the Virtual Fields activity, a subsequent attempt to open the employees record will fail because the lookup into the Virtual Fields table will now fail since the key was not affected by the update. Appendix D: Using Virtual Fields with Virtual Documents 385 Key Initialization The concept and process of initializing keys with Virtual Fields activities usually involves the creation of “key documents” which are native Notes documents stored in the Notes database. They typically only contain key field data used by the Virtual Fields activity to access the corresponding external system table row, so there is one key document created for each row in the table. When using Virtual Fields with Virtual Documents activities to produce a composite document as described in the above preceding sections, the notion of key initialization is somewhat inconsistent with the intended functionality. It is usually assumed that the external table to be used with the Virtual Documents activity has already been “virtualized”. See the Virtual Documents chapter for details on external system table virtualization. In this case, the documents which will play the roll of key documents for the Virtual Fields activity already exist. The addition of virtual fields through the Virtual Fields activity simply requires a common key field and table column as described in earlier examples. As you can see, in this scenario there are simply no keys to initialize since the virtual documents already exist. Because of the transparency of Virtual Documents, it is possible to use the traditional Virtual Fields key initialization in conjunction with Virtual Documents. Assuming that both activities have been set up to monitor the same Notes form and have a common field to act as a key for the Virtual Fields activity, starting the Key Initialization process in the Virtual Fields activity will essentially create virtual key documents. As with ‘normal’ key initialization, one virtual key document will be created for each row in the Virtual Fields table. Of course, each new virtual key document will in turn correspond to a newly created row in the Virtual Documents table. That table must have as a minimum one column for each key used by the Virtual Fields activity. Note The Virtual Documents activity must be running during the key initialization, or else traditional non-virtual documents will be created. This above technique illustrates the flexibility of using Virtual Documents. With this method, you can use an existing table used by a Virtual Fields activity to essentially populate another table mapped to a Virtual Documents activity and thus creating virtual key documents. However, as discussed earlier in this chapter, using virtual documents solely as a way to remove native key documents is not necessarily a good idea, unless it is vital to keep the size of an NSF at a minimum and to keep all data external to the Notes database. However, if this is the intent, a better solution would be to remove the Virtual Fields activity from the picture all together, and just virtualize the existing Virtual Fields table using only Virtual Documents activity to monitor the Notes form. 386 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Automatic Background Virtualization and Key Synchronization Problems Virtualization is the process used by a Virtual Documents activity to adapt an external system table in such a way as to make each row represent a Notes Document. This occurs automatically and immediately when creating a new virtual document through a Notes client . The corresponding row is inserted into the external table when the document is saved. Virtualization can also be done automatically by a background process which scans the external table for existing rows that have not been virtualized as Notes documents, and thereby makes them accessible through Domino. This feature is useful when the external data table already contains data and you want to make the data automatically accessible through Domino. The external data source table is also periodically scanned for rows that may be added through a non-Notes client or process and needs to be virtualized into the NSF. See the Virtual Documents activities chapter for more details. If a Virtual Fields activity is also monitoring the same form, virtualization does not have a direct impact on the Virtual Fields activity or the external table associated with it. However, as described earlier, virtual document must be a common data field in the Virtual Documents activity that also serves as the key for the Virtual Fields activity. If an external table is virtualized and it contains rows with key values that do not exist in the Virtual Fields table, an error will occur when trying to subsequently access the virtual document because the lookup on the Virtual Fields table has failed. If the problem is simply with incorrect key data, leave the Virtual Documents activity running but shut down the Virtual Fields activity. You will then be able to either delete the offending virtual documents or open and update the virtual documents with the correct key data, then restart the Virtual Field activity. Conversely, if the problem is with incorrect key data in the Virtual Fields table, you will have to manually update (for example, through a SQL client) the external data source table with the correct key data, or add the necessary rows, to match the data in the Virtual Documents table. If you have a situation where some of the records in the Virtual Documents table simply do not have applicable data in the Virtual Fields table you may need to make special provisions. For example, the Virtual Documents table contains all company employee data, and the employee’s department code is used as a key to lookup department information in the Virtual Fields table. However, some employees are independent contributors and don’t have a department number (it’s NULL in the Virtual Documents table). This will clearly cause a problem when accessing this document since the lookup on Virtual Fields table will fail. Appendix D: Using Virtual Fields with Virtual Documents 387 Again, it is important to choose a data value(s) in the Virtual Documents table that can always serve as a unique key(s) for the Virtual Fields table. An easy fix for this example might be to simply add a row to the department information table (Virtual Fields table) which has a special department number key (0 for example), with all other department information set to null. Then assign the special department number to each of the applicable employee records in the employee information table (Virtual Documents table). Another more sophisticated approach could be to simply use a post open formula for the Virtual Documents activity which always sets the department number field to 0 if it is null when opening the virtual document. As mentioned above, when creating a virtual document through a Notes client, the corresponding external row will be automatically inserted and virtualized into the external table by the Virtual Documents activity when the document is saved. A new row will also be inserted into the Virtual Fields table for the fields mapped by the Virtual Fields activity if it is monitoring create events. Since the two operations occur together, there should be no synchronization issues. If the Virtual Fields activity is not monitoring create events, it is assumed that the entered key already exist in the Virtual Fields table. Error Conditions and Key Field Synchronization Problems Assuming the external table used by the Virtual Documents activity and the table(s) used by the Virtual Fields activity(s) are initially synchronized; every row in the Virtual Documents table contains a valid unique key(s) for the Virtual Fields table, problems can arise when certain error conditions are encountered. However, problems will occur only when the Virtual Fields activity is monitoring any of the events, which include Create, Update, and Delete events. Depending on your particular write application, you may or may not be using the Virtual Field activity(s) in a read/write fashion. If you are using the Virtual Fields activity in a read-only fashion by only monitoring the Open event, the error conditions discussed in this section are not applicable. See the section “Managing the Key Field Under Different Scenarios” for more information. When creating or updating a document, if an error occurs inserting or updating a record into the Virtual Fields or Virtual Documents table, the document is not created and an error message is returned. For example, if connectivity is lost between Domino and the external system for the Virtual Fields table, a connectivity error will be provided in a dialog box and the error will be logged in the Virtual Fields activity log. The Virtual Document activity processing will not occur and no records will be 388 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide inserted/updated in the Virtual Document table. All Virtual Fields activity processing is performed before Virtual Document activity processing when inserting, updating , or deleting virtual documents. Again, this can only occur if the Virtual Fields activity is monitoring one or more of these events. A problem may occur if an error occurs in the Virtual Document processing, which always occurs after all Virtual Fields activity(s) processing is finished. The result of this is that the Virtual Field table may be updated, or a new row inserted, or a row deleted, depending on the event. However, the corresponding operation fails on the Virtual Documents table, perhaps due to a similar connectivity issue as in the earlier example. At this time there is no rollback on the Virtual Fields table operation. The following describes how to handle this situation, and assumes a single Virtual Fields table with a single key. 1. If creating a new document the Virtual Fields table will contain a new row. When the error condition is resolved, an attempt to create the document may still fail because the Virtual Fields activity will try to reinsert the row with the same key that already exists, resulting in a duplicate key error. To resolve this problem, first delete the new row in the Virtual Fields table or shut down the Virtual Fields activity, prior to attempting to create the document again. After, restart the Virtual Fields activity if you shut it down. 2. If updating a document the problems are similar to creating a new document. The Virtual Fields activity will first update its table with any modified data. A synchronization problem arises only when key field updates are allowed by the Virtual Fields activity (they are blocked by default), and an error occurs in updating the Virtual Documents row with the new key value. A subsequent attempt to open the Virtual Document will fail because it will try to access the Virtual Fields table with what is now a nonexistent key, because it has already been changed in the Virtual Fields table but not in the Virtual Documents table. To rectify this situation, shut down only the Virtual Fields activity, open the document and update the key value field with the correct key, then save and close the document Restart the Virtual Fields activity and reopen the document you should now see all the correct data. 3. If deleting a document the Virtual Fields activity will delete the corresponding row in the Virtual Fields table. When the error condition is resolved, you will be able to successfully delete the document and no error will be reported as a result of the Virtual Fields table row already being deleted. However, an attempt to open the document after the initial error has occurred will result in an error, since the Virtual Fields record has already been deleted so the lookup on that record will fail. Appendix D: Using Virtual Fields with Virtual Documents 389 Non-Key Data Fields Now that the perils of the Virtual Field key field(s) and how to set up the activities with special regards to the key(s) has been explained, how should other non-key data fields be handled? As discussed the only fields that must be mapped in both activities are the fields which will act as the key(s) for the Virtual Field Activity(s). Of course this means these fields must have corresponding columns in both the Virtual Fields and Virtual Documents tables. Beyond this, no other fields need to be mapped in both activities. Whether or not other data fields need to mapped depends solely on your particular application and how the external system tables are set up and used. Using an earlier example where the employee information was stored in a virtualized Virtual Documents table, and look-ups were used into a Virtual Fields table to access the employee’s department information, this would illustrate a case where there would typically be no shared data between the two tables, except for the department code key. The department code key field would be mapped by both activities. Considering the example where the Virtual Fields table contains job performance data for a particular employee, using a unique employee id (for example, social security number) as the key, you might have a different situation since you now have the previously discussed one-to-one relationship between records in the two tables. As such, there might be duplicated information between the two tables. For example, the employee’s full name and telephone number might be included in both tables. If this information, say the phone number, needs to be updated but it is only mapped by one activity, only that activity’s table will be updated, and unexpected results may occur. For example, if the phone number is updated but the phone number field is only mapped by the Virtual Documents activity only the Virtual Documents table will be updated. The Virtual Fields table will not be updated with the new number. From the perspective of the Notes application this does not pose a problem since the unmapped phone number column in the Virtual Fields table will never be accessed. However, you may have some other external application which depends on all the information in the table being used by the Virtual Fields activity to be correct and up to date. As a general rule of thumb, any common data column which is mapped in one activity should also be mapped by the other activity unless the field is a read only field. Be aware that any PRE or POST-event formulas which are run in one activity will not automatically apply to another activity. For example, if a pre-update formula is run to always validate and pre-pend an area code to the phone number field in the Virtual Fields activity, it will have to be duplicated in the Virtual Documents activity to achieve the same results. 390 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Fields Data Storage Option The purpose of this option for ‘regular’ Virtual Fields activities is to simply allow the storage of non-key field data in the NSF, in addition to the external system table. Typically, only the key fields are stored ‘natively’ in the NSF. Setting this option to “leave all (or selected) real-time fields in documents” allows the specified, or all, external non-key fields to be potentially viewed when the activity is not running, or more typically to allow non-key data fields to be displayed in a Notes view. Virtual fields cannot be displayed in a Notes view unless they also stored in the document. Since key fields must always be stored in the document, they can therefore always be displayed in a view. Since here we are using virtual documents, not native key documents, there is no longer any native storage of data in the NSF. However, the limitation on non-key virtual fields that prevents them from being displayed in Notes views still exists. To allow them to appear in views, set the Data Storage option to “leave all (or selected) real-time fields in documents”. In this case of course, those “documents” are virtual documents, so there is an additional requirement that any fields specified in this option must also be mapped in the Virtual Documents activity. This of course means there must also be corresponding columns in the Virtual Documents table. This may not always be feasible or desirable. You can limit the fields to just the selected fields that will be displayed in views, and only those fields then need to be mapped in the Virtual Documents activity. If there are any non-key fields that are otherwise mapped by both the Virtual Fields and Virtual Documents activity for other reasons, and the Data Storage option is set to “remove all real-time fields from the document”, updates to those fields will be set to null in the Virtual Documents table. If you wish to have updates to those fields occur in both tables, you must enumerate them in the Virtual Fields activity Data Storage option to “leave selected real-time fields in documents”, or simply set the option to “leave all real-time fields in documents”. Also, if you don’t do this, you may also receive errors from the external system if nulls are not allowed for any of the fields involved. To summarize, since the virtual documents themselves are fully virtualized Notes documents, all the fields mapped by the Virtual Documents activity can always be displayed in any Notes view. However, any virtual fields mapped by the Virtual Fields activity cannot be displayed in the view unless the Data Storage option is set to “leave all (or selected) real-time fields in documents”, and those fields must also be mapped by the Virtual Documents activity. Appendix D: Using Virtual Fields with Virtual Documents 391 Virtual Field Form Override Option This Virtual Fields activity option allows a Virtual Fields activity to monitor all forms in an NSF. Because Virtual Documents activities always monitor only a single form, this option should generally not be enabled (it is disabled by default) if you are trying to use a Virtual Fields activity to monitor the same form as a Virtual Documents activity. There may be special advanced circumstances where enabling this option may be desirable. Enabling this option may lead to a mix of virtual and ‘native’ documents, all of which may contain virtual fields mapped by the same Virtual Fields activity. For example, creating a document using a form monitored by a Virtual Documents activity will create a virtual document with a corresponding insert into the Virtual Documents table, and a row will also be created in the Virtual Fields activity table. However, creating a document using a form not being monitored by a Virtual Documents activity will result in a native document being created, with the Virtual Fields activity inserting a row into the Virtual Fields table for the virtual fields mapped by the activity. Virtual Attachments Option The Virtual Attachments option is available in both Virtual Documents and Virtual Fields activities. It allows the storage of attachments into an external system table instead of within the NSF. When using a Virtual Fields activity to monitor the same Notes form as a Virtual Documents activity, this option must only be enabled in Virtual Documents activity. By enabling Virtual Attachments in the Virtual Documents activity, all attachments in all the form’s rich text fields will be virtualized into the external system’s table. The rich text field(s) in which the attachments are placed can be mapped by the Virtual Documents activity, or by the Virtual Fields activity(s). In other words, even though Virtual Attachments are enabled through the Virtual Documents activity, the rich text field which contains the link(icon) to the attachment can be mapped by any of the activities monitoring the form. 392 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Appendix E Using Virtual Activities in a Domino Cluster This appendix describes how the Virtual Fields and Virtual Documents activities support the use of Domino clustering. It is assumed that you are familiar with these two activity types. Virtual Activities and Domino Clusters Virtual Documents activities and Virtual Fields activities support Domino clusters but to a varying degree. Virtual Fields, can fully support the failover and load balancing capacity provided by a Domino cluster. Although Virtual Documents will work in a Domino cluster, it currently does not fully take advantage of these capacities. The following describes how to set up and use Virtual activities within a Domino cluster and the limitations. A Domino cluster is a set of specified Domino servers that host one or more replica databases (replicas) and for which Domino cluster replication is enabled. It's assumed that you are familiar with the configuration and use of Domino clusters. All the replicas in the cluster are kept in synchronization using an essentially event driven replication model. Modifications to data in any replica database in the cluster is almost instantly replicated to the other replicas within the cluster. If any server, and therefore its replica databases, were to go off line for any length of time, traffic is automatically routed to the other servers. When the server comes back online, it will be able to replicate with the other servers to bring its replica databases "up to date" with the current data. Load balancing among the servers in the cluster is achieved using a similar mechanism. 393 Virtual Fields Support for Domino Clusters For Virtual Fields activities to fully participate in a Domino cluster, each server in the cluster must be set up with a Virtual Fields activity(s) which monitors the replica database(s) on that server. In other words, each server in the cluster will have its own Virtual Fields activity(s) running against its local replica databases. Each server in the cluster must maintain it's own copy of the LEI Administrator, with a copy of the activity(s) set up and running from each LEI Administrator database. The LEI Administrator cannot be cluster-replicated; each server must host it's own non replica copy of the LEI Administrator. All the Virtual Fields activities can be set up to use the same external data source. If set up in this fashion, cluster failover and load balancing support derives from the fact that each Virtual Fields activity has access to, and can update, the external data. Even if all servers but one were to go off line, the external data would continue to be accessible and can be kept current. As other servers come back online, the current external data would be instantly available to those servers directly from the external data source. 394 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Virtual Documents Support for Domino Clusters The main difference between Virtual Documents support for Domino clusters and Virtual Fields support for Domino clusters stems from the limitation that a single external data table can map to only a single Domino database/form. While it is possible to have a Virtual Documents activity monitoring each replica database in the cluster, you cannot map each of those instances to the same external table, in the manner that would typically be done with Virtual Fields. As a result, the typical configuration calls for selecting a single Domino server in the cluster which will manage the virtual documents and the external data connectivity. A Virtual Documents activity will monitor a replica database and form on this one server, mapping its external data. This activity is responsible for all external data access and updates for the cluster. Other replicas in the cluster will see this virtual data and it will be replicated throughout the cluster in the normal manner, but will be held natively in the other replica databases. Only one replica in the cluster will be running a Virtual Documents activity and "contain" virtual documents. The other replicas will contain native replica documents. In this topography, the external data source becomes just another node in the cluster, and can be detached from the cluster if the server running the Virtual Documents activity becomes disconnected. However, like any other node, data changes made to cluster replicas will be propagated to the server, and ultimately to the external data source, when the server comes back online. Appendix E: Using Virtual Activities in a Domino Cluster 395 Index A Accessibility tools, 11 Activities action bar, 43 Activity Document fields, 59, 60, 62, 65, 67 Admin Backup, 53, 85 Admin Purge Log, 89 Advanced RealTime, 69, 71, 74 Archive, 93 Command, 101 command line execution, 75 data mapping, 76 dependent, 128 description of supplied, 7 Direct Transfer, 77–78, 109, 311, 321 Direct Transfer, output parameters, 118 Direct Transfer, stored procedures, 118 Java, 280 list of supplied, 57 log, 51 logging, 60 Polling, 121, 321 Replication, 77, 251, 321, 339 scheduling, 62, 64–65, 67 Scripted, 53, 283, 287, 291, 321 Virtual Agents, 70, 237 Virtual Documents, 70, 179 Virtual Fields, 69, 131 Admin Backup Activity, 7 Admin Purge Logs Activity, 7 Administrator configuration document, 48 Advanced RealTime activities description, 8 Domino server cluster, 15 specify Domino server, 50 Virtual Fields introduction, 131 with stored procedures, 311 Agent open, 339 Archive Activity, 7 B Browsing buffering, 46 connections from activity document, 58 Domino directory, 45 error messages, 58 fields, 58 in Advanced RealTime activities, 312 names.nsf, 44 server-side, 45 with Virtual Agents, 240 C Cluster, 28 Domino and Advanced RealTime, 15 with iSeries, 16 Command Activity, 7 Connections connectivity, 45, 58 Connectors array transfer, 14 File, 107 list of supplied, 6, 55 Lotus Connector for DB2, 118, 310 Lotus Connector for Files, 119 Lotus Connector for OLE DB, 141 Lotus Connector for SAP, 118 support for output parameters, 175, 177 contents using required Virtual Attachment field, 364 Customer support, xii D Data mapping, 73, 76, 97, 119, 180, 200 columns in Virtual Fields, 135 in activities, 76 Virtual Fields, 141 Data transfer across platform, 81 Data types indexing, 135 LEI, 293 NULL, 295 with Virtual Documents, 187 DB2 Connector identity column support, 76 stored procedures and output parameters, 78 DB2/400, 311 dbib using required Virtual Attachment field, 364 decsadm.nsf, 27, 28 Deletion stubs, 193 DestTimeStamp, 254 Direct Transfer Activity, 7, 311, 321, 327 tutorial, 323 Documentation list, 36, 322 Lotus Developer Domain (LDD), 9 PDF files, 9 www.notes.net, 27, 29, 36 Domino LEI addin task, 21 Domino cluster integrated credentials, 217 with Virtual Documents, 393 with Virtual Fields, 393 Domino Designer, 339 Domino directory, 44, 313 browsing, 45 Domino server LEI Advanced RealTime, 50 397 E EIMODIFIED, 189, 210 EINOTEID, 210 EINOTEPROPS, 182, 186, 188, 192, 210, 212 EIUNID, 191, 210 Notes document unique identifier, 180 Encryption, 29, 134, 182 public key fields, 192 External data system saving data to Virtual Fields, 366 External key table, 191–192 considerations, 213 required column data types, 210 with Virtual Documents, 186 with Virtual Documents activity, 209 writeback, 214 F fileid using required Virtual Attachment field, 364 filename using required Virtual Attachment field, 364 filesize using required Virtual Attachment field, 364 Filter formulas in Virtual Fields, 366 fixup and Virtual Attachments, 164, 218 H HTTP, 310 HTTP client Virtual Agents, 249 HTTP server with Virtual Fields, 135 I Indexing field list, 192 in Virtual Attachments to increase performance, 165, 219 in Virtual Documents, 187, 191 key fields, 135 performance, 14 performance in Advanced RealTime activities, 15 rebuild using Shift + F9, 189 Virtual Documents, 214 Virtual Fields, 134 Initialize keys in Virtual Fields, 157 Integrated credential with Virtual Agents, 249 Integrated credentials, 71, 74 full text indexing, 216 in a Domino cluster, 217 in Virtual Agents, 248 in Virtual Documents, 215 in Virtual Fields, 161, 169 server to server replication, 216 with Virtual Documents, 161 iSeries console commands, 23, 24 job logs, 310 language support, 17 LEI server, 23 qnotes, 17 terminology, 17 tips for use with LEI, 16, 310 troubleshooting, 308 J Java Activity, 7 java.exe, 281 JVM, 281 K Key documents, 133–134, 182, 379, 381 Virtual Fields, 132 Key fields indexing, 135 Key initialization, 386 L lccon6.nsf, xi, 29, 55 Lotus Connectors and Connectivity Guide, 36 LCConnection class, 284 LCSession class, 284 LEI Activities Log activity, 51 LEI Activity Log, 8 LEI Administrator Activity views, 33 author privileges, 29, 44 configuration document, 47 configuration view, 34 Connections view, 34 introduction, 5, 27 user assistant, 9, 35 user documentation, 9 LEI Script Vault, 53, 283, 338 LEI server, 28, 39 command to invoke, 19 commands, 20 console commands, 20 iSeries, 23 log, 8 partitioned, 19 leiact.exe, 75 leiadm.ntf, 28 leicgi.exe, 75 leiclean script, 22 leicred.ntf, 71, 74, 161, 216, 249 leidoc.nsf, xi Lotus Enterprise Integrator Activities and User Guide, 36 leiempsamp.nsf, 13, 322–323, 330, 332–333, 346, 348–349 leiig.nsf, xi, 27 Lotus Enterprise Integrator Installation Guide, 36 leilog.nsf, 8, 28, 51 leipackagetrack.nsf, 53, 283, 321–322, 339, 351–352, 358 leivlt6.nsf, 53, 283, 288, 339 supplied agents, 321–322 lsxlc6.nsf, xii, 29 Lotus Connectors LotusScript Extensions Guide, 36 M Memdata, 182 Metaconnectors list of supplied, 6 Multi-value data in Virtual Fields, 158 N nlei.exe, 19 nleiact.exe, 75 noteid using required Virtual Attachment field, 364 398 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide Notes Agent, 288 notes.ini, 107 EICenturyBoundary, 295 ServerTask, 21 NT service LEI, 22 NULL, 134, 141, 182, 193, 200 in Virtual Fields and Virtual Documents, 135 O Order Metaconnector replication sort order, 256 Output parameters, 109, 118 in Virtual Fields, 171 P Passwords encryption, 28 PDF files, 58 Performance array transfer, 14 database index, 14 Direct Transfer, 13 in Direct Transfer activity, 117 indexing in Virtual Documents, 191, 214 indexing Virtual Documents, 187, 219 indexing Virtual Fields, 135, 165 Replication, 13 Polling considerations, 128 interval, 121 scheduled, 128 timestamp, 123 trigger-based, 123 Polling Activity, 8, 321 tutorial, 327 R readme.txt, xii RealTime see Virtual Fields, 69 Replication conflict handling, 265 key fields, 262 logging, 265 primary key, 251, 265, 311 restrictions, 274 skip deletions, 273–274, 278 skip insertions, 273–274, 276 skip updates, 273–274, 277 sort order and key fields, 256–257 timestamp, 251, 253–254, 256, 263, 271, 311 timestamp results, 270, 274 Replication Activity, 8, 321 tutorial, 332 Replication id and Virtual Agents, 240 S Sample database leipackagetrack.nsf, 283 Sample databases, 322, 323 Scheduling example, 129 restrict to schedule, 128 start and stop times, 128 Scripted Activity, 8, 321 tutorial, 338, 346 Security ACL, 28 in Advanced RealTime activities, 312 integrated credentials database, 71, 74 LEI Administrator, 28 password encryption, 29 Virtual Documents, 182, 191 Virtual Fields, 134 SrcTimeStamp, 254 Stored procedures, 109, 118, 311 examples used in Virtual Fields, 371 in Advanced RealTime activities, 313 in Virtual Fields for Sybase and SQL, 369 mapping, 73 testing DB2 in Virtual Fields, 373 testing SQL in Virtual Fields, 371 using Oracle and Virtual Fields, 375 with Virtual Agents, 238 with Virtual Documents, 229 Stored procedures and output parameters in DB2, 78 Stub documents, 133–134, 182 Virtual Fields, 132 Subkeys use caution when updating, 161 T Table creation utility Create virtual attachment table in Virtual Documents, 220 Create virtual attachment table in Virtual Fields, 167 Terminology agent, 338 common LEI terms, 10 conflict, 265 correlating Notes and relational database, 293 iSeries, 17 LEI Administrator, 5 LEI server, 5 Timestamp polling, 123 Trigger-based polling, 123 Troubleshooting on iSeries, 308–309 using activity logs, 307 Tutorials introduction, 321 U UNID as key field, 141, 150 V Virtual Agents, 70 example, 238, 244 HTTP client, 249 integrated credentials, 248 introduction, 237 options description, 246 stored procedures, 238 used with Virtual Fields, 178 using a URL, 249 Virtual attachment table with Virtual Documents activity, 220 with Virtual Fields activity, 166–167 Virtual attachments columns, 163 in Virtual Documents, 164, 218 in Virtual Fields, 163 indexing fields to increase performance, 165, 219 Virtual Attachments Activity example, 364 Virtual Documents, 70 Index 399 connection security, 191 control data column names, 186 create external key table, 220 data types, 187 Deletion stubs, 193 in Domino cluster, 393 integrated credentials, 215 monitored forms, 183 replication ID, 196 stored procedures, 229 with external key table, 186, 209, 210 with Virtual Fields, 132, 180, 377–378, 380, 387 Virtual Documents Activity, 8 example, 363 Notes Application using Virtual Attachments, 359 Virtual Fields, 69 create external key table, 167 example, 136, 147 in Domino cluster, 393 indexing, 134 initialize keys, 157 integrated credentials, 161 key documents, 132 mapping key and data fields, 149 monitor order, 132 multi-value data, 158 option description, 169 stub documents, 132 used with Virtual Agents, 178 with Virtual Documents, 132, 180, 377–378, 380, 387 Virtual Fields Activity, 8 accessing different external sources, 366 building views, 367 data storage options, 367 example, 362 example query Notes or Web clients, 351 filter formula, 366 monitor orders, 367 Notes Application using Virtual Attachments, 359 saving data to external system, 366 static copy of external data, 367 testing stored procedures in SQL, 371 using an Oracle stored procedure, 375 using stored procedures in Sybase/SQL, 369 Virtualizing existing data, 184 W Web sites accessibility, 11 DECS, xii LEI, xii LEI documentation, xii PTFs, xii Writeback, 214 www.notes.net, 9, xii, 55 400 IBM Lotus Enterprise Integrator for Domino (LEI) Activities and User Guide