Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
IBM Campaign Version-independent Integration with IBM Engage Version 1 Release 3.1 April 07, 2017 Integration Guide IBM Note Before using this information and the product it supports, read the information in “Notices” on page 109. This document describes procedures and features that are supported by IBM Campaign version 8.6.0.7 to 9.1.2.x. © Copyright IBM Corporation 2014, 2017. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Chapter 1. Overview . . . . . . . . . 1 What you can do with the integration . . What you cannot do with the integration . New in this release . . . . . . . . Resources to support the integration . . . . . . . . . . . . . . . . . . 3 3 3 4 Chapter 2. Planning. . . . . . . . . . 7 Security considerations for Campaign and Engage . Security in Campaign . . . . . . . . . Security in IBM Engage. . . . . . . . . Data management for Campaign and Engage . . Integration database tables, ETL, and partitioning . . . . 7 7 8 8 9 Chapter 3. Prerequisites . . . . . . . 11 Integration prerequisites for Engage . . . . IBM Marketing Cloud organization account Creating the integration user . . . . . SMS enablement . . . . . . . . . . Mobile push enablement . . . . . . . Integration prerequisites for Campaign . . . IBM Campaign installation requirements . Integration prerequisites for UBX . . . . . Integration prerequisites for the UBX Toolkit . . . . . . . . . . . . . . . . . . . 11 11 11 14 14 15 15 15 16 Chapter 4. Installation and configuration . . . . . . . . . . . . 17 Installing the integration files . . . . . . . CS_HOME directory . . . . . . . . . Upgrading an existing integration . . . . . . Configuring access to JDBC . . . . . . . . Creating tables for data download . . . . . . Configuring connections between Campaign and Engage . . . . . . . . . . . . . . . Firewall preparations . . . . . . . . . Configuring data exchange properties . . . Configuring access to Engage . . . . . . Proxy server connections . . . . . . . . . Connecting through an HTTP proxy . . . . Connecting through an FTP proxy server . . Encrypting the integration configurations . . . Creating a custom encryption key . . . . . Removing encryption from configuration properties . . . . . . . . . . . . . UBX Toolkit installation . . . . . . . . . Preparing UBX endpoints for the integration . . . . . . . 17 18 18 19 19 . . . . . . . . . 20 21 21 22 22 23 24 24 26 . 26 . 27 . 27 contactUpload . . . . . . . . . . . tableUpload . . . . . . . . . . . . Upload preparation and requirements . . . Uploading data by running a script . . . . Uploading data by running a flowchart . . . Flowchart design for data upload to Engage . Requirement to match Campaign fields to Engage fields . . . . . . . . . . . . . . . Mapping field names in a flowchart . . . . Mapping fields with a custom XML file . . . Requirement to match date formats . . . . . Default date format for uploads . . . . . Uploading dates with derived fields . . . . . . . . . . 33 36 38 38 38 39 . . . . . . 43 44 45 46 46 47 Chapter 6. Contact data download Engage to Campaign. . . . . . . . . 49 Contact data available from Engage . . . . . . Where to download contact data . . . . . . . How to download contact data . . . . . . . . contactDownload . . . . . . . . . . . Downloading contact data by running a script. . Scheduling contact data downloads by running a flowchart . . . . . . . . . . . . . . Flowchart design for downloading contact data 49 50 50 50 51 52 53 Chapter 7. Tracking data download Engage to Campaign. . . . . . . . . 57 Tracking data provided by Engage . . . . . Adding additional tracking information . . . Tracking data available through the UBX Toolkit Where to download tracking data . . . . . . How to download tracking data . . . . . . trackingDownload . . . . . . . . . . Downloading tracking data with a script . . Scheduling tracking data downloads . . . . Tracking data download with the UBX Toolkit. Error reporting for tracking data . . . . . . . 57 . 58 59 . 60 . 61 . 61 . 62 . 63 . 63 . 65 Chapter 8. Email messaging . . . . . 67 Uploading email contact data to Engage . . . Sending email messages after uploading contact data . . . . . . . . . . . . . . . Downloading email contact data . . . . . Downloading email tracking data . . . . . Support for Send Time Optimization . . . . Filter mailing templates . . . . . . . . . . 67 . . . . . . . . . . 70 72 73 74 74 Chapter 5. Data upload - Campaign to Engage . . . . . . . . . . . . . . 29 Chapter 9. SMS messaging . . . . . . 77 What to upload from Campaign . . . . . . . 30 Where to put uploaded data in Engage . . . . . 30 Finding contact list, database, and relational table IDs in Engage . . . . . . . . . . . . 31 Finding mailing IDs in Engage . . . . . . . 32 How to upload data to Engage . . . . . . . . 32 General requirements for SMS messaging . . . . Consent requirements for SMS messaging . . . . Data required for SMS consent . . . . . . . Uploading SMS contact and consent data to Engage Sending SMS messages after uploading SMS contact and consent data . . . . . . . . . . . . © Copyright IBM Corp. 2014, 2017 77 78 78 79 80 iii Downloading SMS consent data . . . . . SMS Opt-in and Opt-out synchronization between Campaign and Engage . . . . Downloading SMS message tracking data . . Chapter 10. Mobile app messaging . . 81 . . . 82 . 83 . . 85 Mobile push contact synchronization between Campaign and Engage . . . . . . . . . Uploading mobile push contact data to Engage . Sending push notifications after uploading contact data . . . . . . . . . . . . . . . . Downloading mobile push contact data . . . . Downloading mobile push tracking data . . . . 85 . 86 . 86 . 87 . 87 Reasons for contact suppression SP_Attachment . . . . . . SP_BounceReply. . . . . . SP_Click . . . . . . . . SP_Conversion . . . . . . SP_Forward . . . . . . . SP_Open . . . . . . . . SP_OptIn . . . . . . . . SP_OptOut . . . . . . . SP_Sent . . . . . . . . SP_Suppressed . . . . . . SP_UploadAudit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 . 97 . 98 . 98 . 99 . 100 . 101 . 102 . 103 . 104 . 104 . 105 89 Before you contact IBM technical support . . . . . . . . . . . . . . 107 Chapter 12. Custom tracking tables . . 95 Notices . . . . . . . . . . . . . . 109 Event types . Report IDs. . Trademarks . . . . . . . . . . . . . Privacy Policy and Terms of Use Considerations Chapter 11. Reference: Mapping files iv . . . . . . . . . . . . . . . . . . . . . . . . . 95 . 96 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide . 111 111 Chapter 1. Overview The integration of IBM Campaign with IBM Engage combines the precision segmentation tools in Campaign with the robust, cloud-based messaging capabilities of IBM Engage. IBM Engage is part of the IBM Marketing Cloud. Use the integration to upload Campaign segmentation and contact data to Engage databases, contact lists, and relational tables. You can then apply the information from Campaign and messaging tools in Engage to target specific audiences, individually personalize each message, and reach current and potential customers through email, SMS, and mobile push notifications. When the message recipients respond to Engage, you can download contact and tracking data back to Campaign to retarget message responders so that you can maintain contact throughout the customer journey. How it works The integration is based on a package of scripts that interact with Campaign flowcharts and Engage application programming interfaces (API). Download and configure the integration package to enable a secure, automated exchange of segmentation, contact, and tracking data between Campaign and Engage. Run the data upload and download scripts from a command line or automate the data exchange by adding a script to the configuration of a Campaign flowchart. v Configure the integration package to establish the connection between Campaign and Engage. v Design and run flowcharts in Campaign to determine what to send to Engage. v Design and run upload scripts to determine how to send the data to Engage and determine where the data goes. v Run download scripts to bring responses back to Campaign. The integration also takes advantage of IBM Universal Behavior Exchange (UBX) and the UBX Toolkit to bring tracking data from Engage into Campaign. Use the version-independent integration of Campaign with Engage with Campaign versions 8.6.0.7 through 9.1.2. If you install or upgrade to Campaign 10.0, you do not need to use the integration that is described here. The integration at a glance. The following diagram illustrates the various elements of the integration and the flow of data between them. © Copyright IBM Corp. 2014, 2017 1 ▌A▐ Campaign. Select message recipients. ▌B▐ Integration scripts in a flowchart trigger, which you add to a Mail List process. Upload data to Engage. ▌C▐ Engage. Receive data in Engage databases, contact lists, and relational tables. ▌D▐ Use the data from Campaign with Engage message templates to create email, SMS, and mobile app messages. ▌E▐ Send messages in various channels. ▌F▐ Recipients receive the messages and respond. ▌G▐ Return contact and tracking data to Campaign. Depending on channel, use download scripts or UBX and the UBX Toolkit. In UBX, register Engage as an events publisher. ▌H▐ Use the integration download scripts and the UBX Toolkit to receive the downloaded data and add it to Campaign tables. In UBX, register Campaign as an events subscriber. 2 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide What you can do with the integration The integration of Campaign with Engage gives Campaign users a way to use Engage messaging tools and gives Engage users access to segmentation tools in Campaign. You can use the integration software and procedures to perform the following tasks. v Use flowcharts and integration scripts in Campaign to select and automatically upload data to Engage. Use existing Campaign scheduling features to schedule the data upload for a specific time or on a recurring basis. v Upload segmentation data from Campaign and automatically trigger outbound messaging based on pre-configured email, SMS, or mobile push templates. v When you trigger outbound messages with a data upload, you can add a campaign code or other label to the mailing name for better response tracking. v Upload the output of a Campaign flowchart to update the contents of one or more existing contact lists, databases, or relational tables in Engage. You can specify a contact list or database by ID or by name. v Define a new contact list in Engage when you upload the output of a Campaign flowchart. v Use flowcharts and scripts in Campaign to select and upload SMS contact and consent data to Engage. In Engage, send SMS messages that are addressed and personalized with the uploaded segmentation data. v Use integration download scripts and existing Campaign scheduling features to download contact data and tracking data from Engage. Schedule the data download for a specific time or on a recurring basis. v Use IBM Universal Behavior Exchange (UBX) and the UBX Toolkit to download SMS and mobile push tracking data from Engage. What you cannot do with the integration Some aspects of the integration of Campaign with Engage limit what you can do in each application with the data that you share between them. v You cannot upload offer information from Campaign to Engage. The integration of Campaign with Engage does not support Campaign offers. v You cannot use the integration to send SMS messages until you purchase SMS messaging for IBM Marketing Cloud and IBM provisions your Engage organization for SMS messaging. See Chapter 3, “Prerequisites,” on page 11. v You cannot use the integration to send mobile app messages until you enable your IBM Marketing Cloud account for mobile push and implement your mobile app with IBM Marketing Cloud. See Chapter 3, “Prerequisites,” on page 11. v You cannot use the integration as a data migration tool. The integration is designed only to upload the output of a single Campaign flowchart to Engage and to download selected contact and tracking data from Engage to Campaign. New in this release IBM periodically updates the integration of IBM Campaign and Engage to improve and expand what you can do with the integration of the two applications. This release introduces the following new features and capabilities. v Expanded support for SMS messaging Chapter 1. Overview 3 This release enables you to upload contact data without the need few a custom mapping file. You can send SMS messages automatically after you upload SMS contact data. v Expanded support for mobile push notifications. This release enables you to upload contact data with configurable sync fields. You can send push notifications automatically after you upload contact data. v Configurable sync fields for contact upload to support non-keyed databases in Engage You can define a synchronization field in the integration configuration properties and selectively override the field settings when you run an integration script from the command line or as part of a flowchart trigger. v Send Time Optimization support for Email When you upload contact data to Engage, you can now include a parameter that instructs Engage to enable Send Time Optimization. v Separate proxy server support for HTTP and FTP If your corporate data security plans demand separation between your corporate data and the public Internet, you can connect to IBM Engage through either an HTTP proxy or an FTP proxy. Your choice. v SMTP Authentication for sending error logs As part of the integration configuration, you can now specify the credentials that are required to access the SMTP server that you use to send error reports for tracking download. v Expanded response tracking through the UBX Toolkit You can now download tracking data for all messaging channels through the UBX Toolkit and IBM Universal Behavior Exchange (UBX). Resources to support the integration The Campaign to Engage integration provides various types of support to help you use the products together and resolve problems that might occur. v The IBM Campaign and IBM Engage Version-independent Integration Guide provides descriptions and procedures for concepts and tasks that are specific to the integration. For concepts and tasks that apply to Campaign, Engage, UBX, or the UBX Toolkit separately from the integration, see the general product documentation that is available for each application. v Product documentation for IBM Engage is available in the IBM Marketing Cloud Knowledge Base at https://portal.silverpop.com. You must provide valid login credentials to access the IBM Marketing Cloud Support Portal. v Product documentation for Campaign is available on the IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SSCVKV/ product_welcome_kc_campaign.dita v Product documentation for IBM Universal Behavior Exchange (UBX) is available on the IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/ kc_welcome_UBX.dita 4 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide v Product documentation for the UBX Toolkit is available on the IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/UBX_KC_mapgentopic4.dita v The integration scripts include detailed help on the command line through the -h or -help option. v The property files and example files that are provided as part of the integration contain detailed comments that explain how to customize the integration for your working environment. Chapter 1. Overview 5 6 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 2. Planning The first phase of integrating IBM Campaign with IBM Engage requires satisfying requirements in both product environments for network communication, application access, and data management. To ensure a successful integration in your environment, review and complete all system and security requirements for Campaign and Engage before you install and configure the integration files. Consider the effort, user permissions, and resources that are required to satisfy these requirements in your working environment. Security considerations for Campaign and Engage IBM Campaign and IBM Engage control application access in slightly different ways. Planning how to control access to the integration of Campaign and Engage requires an understanding of both products and how they can operate together. As you plan to implement the integration of Campaign and Engage in your environment, consider the following issues. v User accounts, roles, and permissions that are defined in Campaign are separate from users and permissions that are defined in Engage. There is no single sign-on between the two products. Marketing users require separate user accounts to access Campaign and Engage. v The integration scripts require a special user account so that they can operate automatically. This user is referred to as the integration user. v In Engage, use shared databases instead of private databases so that all members of your organization can access them. Content in private databases is not available to other users or to the integration scripts. Related concepts: “Firewall preparations” on page 21 Security in Campaign IBM® Campaign installs with IBM Platform. Security for both applications is based on user permissions that are set for each application. In IBM Campaign and IBM Marketing platform, you define logins for individual marketing users and set permissions on them. The permissions limit what each user can do within Campaign. You can create various security policies and roles to manage user permissions. You set policies on top level folders within Campaign. the top-level policies, in combination with user permissions, restrict which objects in Campaign users can access and what they can do with them. For more information on security in IBM Campaign, see the IBM Campaign Administrator's Guide. © Copyright IBM Corp. 2014, 2017 7 Security in IBM Engage For more information on security in IBM Engage, see the IBM Marketing Cloud Knowledgebase. Related tasks: “Configuring access to Engage” on page 22 Data management for Campaign and Engage To use Campaign and Engage together, you must understand where your data lives. You need to know what your IT security and privacy officers are comfortable uploading to the Cloud and publishing in email. You also need to anticipate the types of segmentation and personalization that you need to effectively reach your customers. The integration of IBM Campaign and IBM Engage depends heavily on the exchange of information between the two applications and how that data exchange is managed. It is important to remember that IBM Campaign is installed locally, behind corporate security firewalls. Engage is a cloud-based service that is accessed through secure login. In Campaign, you can design flowcharts to create rich segmentation over the data in any database to which Campaign is connected. The segmentation includes data from external data imports and activity from multiple channels, such as direct mail, call center activity, and email. You can send the segmentation output to a file or database table, where fulfillment processes for different channels can pick up the list of contacts and use them. The output can also contain other database fields for personalization purposes. Engage mailings and programs work with either queries or contact lists. A contact list is a list of rows within an Engage database. The database can be populated manually, through an API, or through web forms. A query is a set of criteria that can be based on any data that is available to Engage, whether in databases or relational tables. Programs in Engage can also use Universal Behaviors to decide whether to add a particular contact to the program. The Campaign integration with Engage allows marketing users to upload or download database data from Engage, upload relational table data, and download tracking events from Engage email tracking. Flowcharts can also add contacts to a contact list based on Campaign segmentation and integration scripts. Some local Campaign data must be uploaded to the cloud in order for Engage to work. The following examples are typical of the kind of data that either exists in the cloud-based systems or must be uploaded to those systems. v Data for keys that you define in your Engage database. v Data that is required to personalize the messages that you send. For example, if you want to address the contact by first name and title in your messages, you need to have first name and title stored in your database. v Any Universal Behavior data that you are planning on taking advantage of in programs. Universal behavior data is stored in Engage and cannot be downloaded to the Campaign database. v Any data that is collected through Engage web forms. 8 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Note: If you plan to use data from Campaign in messages that involve SMS, Mobile Push Notification, CRM integration, or Engage Universal Behaviors you must upload the data to a non-keyed Engage database. Data that does not need to go into the cloud includes any data that is not in the categories that are described above. For example, you might need a postal code to determine whether an individual contact is near a participating retail outlet, or you might want to check purchase history to determine whether an individual is a high, medium, or low value customer. You can use Campaign to make those determinations locally, behind the firewall, and upload only the contact list that results from the query. You do not need to upload the postal code or the purchase history to Engage to use that data in your segmentation criteria. Related concepts: “What to upload from Campaign” on page 30 “Where to put uploaded data in Engage” on page 30 Integration database tables, ETL, and partitioning The Campaign integration with Engage populates different database tables for auditing and tracking. Consult with your database administrator to discuss how long you need to keep the data for querying. Depending on the volume of activity for your account, the tables can grow large over time. Each integration table shares some characteristics. v The primary key is an identity or sequence column. The IDs in the primary keys reflect the order in which rows were inserted. v The tables have a datetime/timestamp column to indicate the time at which a particular event happened. v The rows in each table are inserted once and the integration does not update them after the initial insert. v There are no predefined indexes, foreign keys, or check constraints other than the primary key. If you are not using recipient email address as the audience level in Campaign, you can add one or more columns to the tracking tables. However, your data must include a way to look up audience level for any contact. You must configure the integration to download the values for those columns from your Engage database. When you add columns, do not use unique indexes or constraints because you might prevent data from being inserted. The integration scripts do not automatically purge or archive the tables. Your administrator can schedule archiving or purging of the data. A typical purging scheme might set up range partitioning on the datetime/timestamp field, with partitions for each month or quarter. The purging plan can drop partitions when they become outdated. However, different database capabilities and performance characteristics can affect your strategy for partitioning and purging of data. How you query the data might also affect your strategy. When you download email events by using ETL, Campaign processes email events for populating detailed contact history and response history. Campaign runs these ETL events: Mailing Sent Event and Mailing Bounce Event to populate the Chapter 2. Planning 9 detailed contact history and Mailing Open Event and Mailing Clickthru Event to populate the response history. This ETL process runs in the background of the Campaign web application. You can configure the schedule and frequency for ETL under: "Settings for 'contactAndResponseHistTracking' (Affinium|Campaign|partitions|partition1|Engage|contactAndResponseHistTracking)". As Engage runs ETL events on the Campaign web application, a cluster user can control Engage ETL execution on individual nodes by specifying JVM option: -Dengage.etl.disabled=true. When -Dengage.etl.disabled=true. This option disables Engage ETL on the specified cluster node. 10 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 3. Prerequisites The various components that play a role in the integration of Campaign and Engage each have a set of prerequisites that you must satisfy before you install and operate the integration. You must satisfy each set of prerequisites separately. Information that you might need to complete the prerequisites for Campaign or Engage is included here or in the supporting documentation for each product. Integration prerequisites for Engage Before you can use Engage with Campaign, you must establish access to Engage. You must create an IBM Marketing Cloud user account to gain access on behalf of your organization and configure a system user with access to Engage to perform integration tasks. Learn more about IBM Marketing Cloud and how to request a user account at http://www.ibmmarketingcloud.com/. Related concepts: “Integration prerequisites for Campaign” on page 15 IBM Marketing Cloud organization account IBM Engage is part of IBM Marketing Cloud. To access IBM Engage, you must provide a valid user name and password. You must have an IBM Marketing Cloud account to receive a login. IBM Engage manages user and application access to Engage through organizations that are created in Engage. You can designate one or more members of your marketing or IT team as organization administrators. An organization administrator can configure organization settings, manage users, supervise mailings, and monitor and assess mailing results. The organization administrator is also granted the user permissions within Engage that are required to configure the integration of Campaign with Engage. If you are a current IBM Engage customer, you can use your existing account login to access Engage to configure the integration. If you are a Campaign user who wants to use IBM Engage, your business organization must establish an account with IBM Marketing Cloud. To request credentials to access IBM Engage, contact your IBM representative. Creating the integration user IBM Campaign automatically accesses data management and messaging features in the Engage environment as a standard Engage user over a secure HTTPS connection. In the context of the IBM Campaign and IBM Engage integration, this user is referred to as the integration user. © Copyright IBM Corp. 2014, 2017 11 Before you begin Access IBM Engage to complete this task. You must have a valid IBM Marketing Cloud login to access Engage. You must be an organization administrator to create the integration user. About this task Accessing Engage as the integration user provides IBM Campaign with a way to submit access credentials to IBM Engage without the need for manual intervention. To fully configure the integration user, you must manually log in as the integration user at least once so that you can define a persistent user password. You must configure the integration user to have permissions to upload and download data, add and remove contacts, and to send mailings. You must also associate the user with an email account that can receive system notifications and credentials. Specify an email account that you can access easily and monitor frequently. The integration user does not require access to the IBM Marketing Cloud Support Portal. Procedure 1. In Engage, go to Settings > User Accounts. Click Create User Account. Accept the default settings except as noted in the following steps. 2. On the Create User page, complete the User Information fields. a. In the User section, enter a user name and enter a password that conforms to the password standards that are defined on the page. Select Do not enforce password expiration policies for this user. Note: The password is a temporary password that is valid for 7 days. After you configure the integration user, you must log in to Engage within 7 days to define a permanent password. b. Enter a name for the integration user. The name can be any name. c. In the Role field, select Standard User. d. In the Contact fields, provide contact information that the system uses to provide email notifications. Enter an email address that you can access and monitor readily. Important account credentials are delivered to the address that you specify. e. In the Localization section, enter a time zone and default language. Enter the time zone that is appropriate for where the Campaign Analytic server is located. f. In the Email Defaults section, enter a name, email address, and reply email address that the system uses to define the sender of the marketing email that you send. 3. Click Save. After you save, you can define other aspects of the integration user. 4. In the User Permissions section, select the following options to define permissions for the integration user. v Data access. For this permission, select Add/Remove Contacts from Contact Lists and Edit Queries. v Allow user to upload content to Stored Files and Mailing Templates 12 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide 5. Click Save. 6. Go to Settings > Organization Settings. You must be an organization admin to access this menu. The Organization Settings window opens. 7. In the Application Account Access section, click Add Application. Enter Campaign Integration as the name to identify the integration as an external application that can access Engage data. Click Add. 8. Click Add Account Access. The Add Account Access window opens. 9. In the Application field, select Campaign Integration, which is the name that you assigned for the integration in Step 7. In the User Account field, select the user name that you entered in Step 2. Click Add. 10. Monitor the email notification address that you defined for the integration application until you receive an email notification. The notification contains the application name that you defined the integration application and a refresh token. 11. In the Application Account Access section, click the name that you assigned to the integration application. A Client ID and Client Secret display. Record the values for future use. 12. Log out of Engage. Log back in to Engage with the user name and password that you defined for the integration user in Step 2. a. Go to Settings > User Profile. b. In the User section, enter the current password for the integration user and define a new password. The new password is the permanent password for the integration user. c. Click Save. Results The new integration user displays as a standard user on the Enabled tab of the User Accounts page. Following information is defined for the integration user. v User name v Password v Client ID v Client Secret v Refresh token Record the information in a secure location. The information is required to configure access between Campaign and Engage. If you configured the user to ignore password expiration policies as required in Step 2, you are not prompted to change this password again. What to do next The integration user runs the integration scripts. You must specify the user name and password that you define for the integration user in the sp.properties configuration file. The integration user can operate only on shared objects in Engage. When you create objects in Engage, such as databases, contact lists, or mailings, be certain to create them as shared objects. Chapter 3. Prerequisites 13 SMS enablement To send SMS messages from IBM Engage, you must complete certain preparations to enable your Engage organization for SMS. You must be an organization administrator to complete some of the steps in the process. The Provisioning Team for the IBM Marketing Cloud must enable your organization for SMS. Then, you must log into Engage and complete additional steps to prepare your organization for SMS messaging. v Create and enable an Engage database for SMS. The database must be a non-keyed database. v Configure SMS integration between Engage and the SMS Campaign Manager. Each Engage organization can have only one database that is enabled for SMS. If you also plan to send mobile push notifications you can enable a single database for SMS and Mobile App Messages. However, you can also enable one database for SMS and another database for Mobile App Messages. For more information regarding how to enable your IBM Marketing Cloud organization for SMS messaging, see the description of SMS in the IBM Marketing Cloud Knowledge Base at https://kb.silverpop.com/kb/Engage/mobile/SMS__Silverpop_Mobile_Messaging. Mobile push enablement To send mobile push notifications from IBM Engage, you must complete certain preparations to enable your Engage organization for Mobile App Messages. You must be an organization administrator to complete some of the steps in the process. The Provisioning Team for the IBM Marketing Cloud must enable your organization for Mobile App Messages. Various individuals contribute to making it possible to send mobile push notifications from Engage, including mobile app developers, the Engage organization administrator, and mobile marketers. The entire process includes the following general steps. v The Provisioning Team enables Mobile App Messages for your organization. v Mobile app developers build and register your mobile app. v Download the SDK. Implement the SDK within the mobile app. v Customize the app, as needed. v Create an Engage database for Mobile App Messaging. The database must be a non-keyed database. v Set up the mobile app in IBM Marketing Cloud. Each Engage organization can have only one database that is enabled for Mobile App Messaging. If you also plan to send SMS messages, you can enable a single database for SMS and Mobile App Messaging. However, you can also enable one database for SMS and another database for Mobile App Messages. 14 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide For more information regarding how to enable your IBM Marketing Cloud organization for Mobile App Messages, see the description of the process in the IBM Marketing Cloud Knowledge Base at https://kb.silverpop.com/kb/Engage/ mobile/Mobile_App_Messages. Integration prerequisites for Campaign Before you can use Engage with Campaign, you must update the IBM Marketing Software environment, including the installations of IBM Platform and IBM Campaign. If you are an Engage user who is beginning to use IBM Campaign, you must install and configure IBM Campaign in your local computing environment. Campaign requires that you also install and configure IBM Marketing Platform. v If Campaign is not installed in your local environment, you must install Campaign. v Define Campaign users with appropriate roles and permissions. v Identify the data that you need to upload to Engage. v Download and install the integration package on the Campaign Analytics Server. For a full list of installation and configuration requirements for Campaign, see the IBM Campaign Installation Guide or the IBM Campaign Upgrade Guide. Related concepts: “Integration prerequisites for Engage” on page 11 IBM Campaign installation requirements To use Engage messaging tools with Campaign segmentation tools, you must install and configure IBM Campaign. Campaign requires that you also install and configure IBM Marketing Platform. The IBM Campaign integration with IBM Engage is supported on IBM Campaign version 8.6.0.7 to 9.1.2. For more information about how to install Campaign, see the IBM Campaign Installation Guide. To integrate Campaign with Engage, you must complete the following tasks. v Configure the Campaign system tables and schema. v Define roles and permissions for Campaign users. For information on installing and configuring IBM Campaign, see the IBM Campaign Installation Guide and the IBM Campaign Administrator's Guide. Integration prerequisites for UBX You can use UBX and the UBX Toolkit to download tracking data from Engage. To use UBX you must have a UBX user account. Access to UBX user accounts requires entering a user name and password that IBM creates for each UBX user account. Chapter 3. Prerequisites 15 Contact IBM to request a UBX user account. As part of the installation and configuration process, you register Campaign and Engage as endpoints in UBX and configure event syndication between them. Integration prerequisites for the UBX Toolkit The UBX Toolkit depends on UBX and on files that you install in your local environment. You must establish the UBX account and complete the UBX Toolkit installation and configuration before you can download tracking data for SMS messages and mobile push notifications. To access UBX, IBM must create and provision a UBX account on your behalf. To request that IBM to create and provision your UBX account, contact the UBX Account Provisioning team by email at [email protected] or request access to UBX at https://www-01.ibm.com/marketing/iwm/iwm/web/ signup.do?source=ibm-ubxprovision. Installing and operating the UBX Toolkit requires access to various systems in your local network environment. Establishing this access typically requires network or database administrative privileges. For a full list of installation and configuration requirements for the UBX Toolkit, see the UBX Toolkit Users Guide on IBM Knowledge Center here: http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/UBX_KC_mapgentopic4.dita 16 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 4. Installation and configuration The integration of Campaign with Engage depends on a collection of files that you download from IBM developerWorks to the environment in which Campaign is installed. You install the files on the Campaign Analytics server. You perform all of the installation and configuration tasks, except for firewall modifications, in the local Campaign environment. Installing the integration files To install the IBM Campaign integration with IBM Engage, download the integration package from IBM DevWorks and extract the archive into a directory in your existing Campaign installation. Before you begin Complete the integration planning and prerequisites for Campaign and Engage. Procedure 1. Go to IBM devWorks and download the installation package for your operating system. Table 1. Integration Installation Packages Operating System Installation download package Windows IBM_Campaign_Integration_with_IBM_Engage_1.3_Windows.zip UNIX or Linux IBM_Campaign_Integration_with_IBM_Engage_1.3_Unix-Linux.tar.gz 2. On the Campaign Analytics server, extract the compressed archive into a folder under ../Campaign/partition/partition1/ . The folder where you install the files is considered the <CS_HOME> directory. 3. In <CS_HOME>, configure the setenv file. This file is in the bin directory. a. Copy and rename example_setenv.bat to setenv.bat or example_setenv.sh to setenv.sh. b. Configure the setenv file as instructed by comments in the file. 4. In <CS_HOME>, configure jdbc.properties. This file is in the conf directory. a. Copy and rename example_jdbc.properties to jdbc.properties. b. Configure jdbc.properties as instructed by comments in the file. 5. In <CS_HOME>, configure config.properties. This file is in the conf directory. a. Copy and rename example_config.properties to config.properties. b. Configure config.properties as instructed by comments in the file. 6. In <CS_HOME>, configure sp.properties. This file is in the conf directory. You must be familiar with the configuration of the integration user to complete this step. a. Copy and rename example_sp.properties to sp.properties. b. Configure sp.properties as instructed by comments in the file. © Copyright IBM Corp. 2014, 2017 17 7. In <CS_HOME>, make a copy of example_logback.xml. This file is in the conf directory. a. Rename the copy as logback.xml. b. (Optional) Update the <max History> tag. By default, this tag is set to 14. This setting maintains a log history of 14 days. You can change this value to suit your requirements. 8. For UNIX or Linux environments only, change directory to the bin directory, then run chmod +x *.sh. What to do next Create database tables to accept data that is downloaded from Engage. Related tasks: “Creating tables for data download” on page 19 “Configuring connections between Campaign and Engage” on page 20 CS_HOME directory The directory where you install the scripts, property files, and other software to support the Campaign integration with Engage files is referred to as <CS_HOME>. The integration files must be installed in a folder under ../Campaign/partition/ partition1/, on the Campaign Analytic server. Note the path to the <CS_HOME> folder. You specify an absolute or relative path to this folder when you run the integration scripts. Related tasks: “Downloading tracking data with a script” on page 62 Related reference: “contactUpload” on page 33 “tableUpload” on page 36 “contactDownload” on page 50 “trackingDownload” on page 61 Upgrading an existing integration IBM periodically upgrades the Campaign to Engage integration to fix observed issues and to introduce new capabilities. Before you begin To preserve your current configurations, back up your existing Campaign to Engage integration installation. The upgrade replaces the existing files with new files. Procedure 1. Backup the current <$CS_HOME> directory. You will need to use the backup files to replicate your configuration settings. 18 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide 2. Download the compressed installation file archive and extract the files into <$CS_HOME>. The new files overwrite the existing files. Windows: IBM_Campaign_Integration_with_IBM_Marketing_Cloud_1.3_Windows.zip Linux or UNIX: IBM_Campaign_Integration_with_IBM_Marketing_Cloud_1.3_Unix-Linux.tar.gz 3. In the conf directory, open example_sp.properties. Add the new proxy properties to your sp.properties file. v sp.proxy.url URL for the web proxy server. v sp.proxy.username User name to access the web proxy server. v sp.proxy.password Password to access the web proxy server. v sp.proxy.ftp.url=URL for the FTP proxy server. v sp.proxy.ftp.username= User name to access the FTP proxy server. v sp.proxy.ftp.password= Password to access the FTP proxy server. 4. In the ddl directory, run the appropriate version of the cs_tab_upgrade_1.01.2_xxx.sql script for your database. v DB2: cs_tab_upgrade_1.0-1.2_db2.sql v Oracle: cs_tab_upgrade_1.0-1.2_ora.sql v MS SQL Server: cs_tab_upgrade_1.0-1.2_sqlsvr.sql Configuring access to JDBC Use the jdbc.properties file to configure JDBC access to enable data transfer between IBM Campaign and IBM Engage. About this task To configure properties in this file, you must know the database driver class and the JDBC URL for the Campaign database that contains the data that you upload to Engage. You must also know the name and password for the system user that is used to access the Campaign database. Procedure Comments in jdbc.properties provide guidance for configuring each property. The jdbc.properties file is in the conf directory of CS_HOME. Creating tables for data download To enable Campaign to receive email contact and tracking data from Engage, you must create new database tables to receive the downloaded data. The integration package includes several DDL scripts that you can use to create the required tables. Before you begin Confirm that the integration files are installed in a folder on the Campaign Analytics server. The folder that contains the integration files is considered the <CS_HOME> directory. Locate the ddl folder. Procedure 1. On the Campaign Analytics server, navigate to the ddl folder in the <CS_HOME> directory. Chapter 4. Installation and configuration 19 2. Use your database client to run the DDL scripts against the database or schema that holds the Campaign system tables. v Microsoft SQL server: cs_tab_sqlsvr.sql v Oracle: cs_tab_ora.sql v IBM DB2: cs_tab_db2.sql Running the DDL script against the Campaign database extends the Campaign database schema to include the new tables. Results The following tables are created in the Campaign database schema. Tracking table Description More information SP_Attachment Records when an attachment is downloaded or viewed. “SP_Attachment” on page 97 SP_BounceReply Records of unsuccessful message delivery. “SP_BounceReply” on page 98 SP_Click Records for link clicks in messages. “SP_Click” on page 98 SP_Conversion Records of Engage conversion events. “SP_Conversion” on page 99 SP_Forward Records of messages that were forwarded. “SP_Forward” on page 100 SP_Open Records of message opens. “SP_Open” on page 101 SP_OptIn Records that are marked as opted-in. “SP_OptIn” on page 102 SP_OptOut Records that are marked as opted-out. “SP_OptOut” on page 103 SP_Sent Records of messages sent. “SP_Sent” on page 104 SP_Suppressed Records of recipients for whom messages are suppressed. “SP_Suppressed” on page 104 SP_UploadAudit Record of all data uploads and related API calls. “SP_UploadAudit” on page 105 The SP_UploadAudit table is updated when marketers run the contactUpload or tableUpload script to upload data and segmentation to Engage. The remaining tables are updated when marketers run the trackingDownload script to download tracking data from Engage. Related tasks: “Installing the integration files” on page 17 Configuring connections between Campaign and Engage The Campaign integration with Engage includes various configuration files that you modify to establish and support secure upload and download of contact and tracking data. The configuration files are provided as part of the download package. 20 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide About this task Configuring the connection between Campaign and Engage involves the following steps. v Preparing the corporate firewall. Typically, corporate IT or network administrators perform this task. v Configuring data exchange properties. v Configuring access to Engage. Optionally, you can configure a proxy server between Campaign and Engage. Related tasks: “Installing the integration files” on page 17 Firewall preparations The integration scripts, and the Campaign flowcharts that use the scripts as triggers, run on the Campaign Analytic server, behind your corporate firewall. Network administrators must configure the firewall to allow the Campaign Analytic server to communicate with the Engage servers that are installed in the cloud-based IBM Engage messaging infrastructure. Consulting with your network administrators to plan how to support communication between Campaign and Engage is a critical step in the integration planning process. The integration cannot operate unless the corporate firewall is configured to allow Campaign to securely exchange data with the Engage servers. The integration scripts communicate with Engage over HTTPS and SFTP. To support these communication methods, network administrators must open ports in the firewall as follows. v HTTPS: Port 443 v SSH: Port 22 IBM Marketing Cloud designates specific Engage servers for use by your organization. In the integration configuration, the sp.properties file defines the Engage server addresses. Administrators require these addresses to update the firewall configuration. The sp.properties file is in the conf directory. The sp.properties file contains the sp.ftp.port setting that defines the SSH port through which the integration transmits and receives data over SFTP. By default, this setting is set to port 22. Comments in the sp.properties file explain how to configure the various configuration settings. Related concepts: “Security considerations for Campaign and Engage” on page 7 Configuring data exchange properties Use the config.properties file to configure and manage data exchange between IBM Campaign and IBM Engage. You can also configure properties in this file to specify how the system reports and responds to errors. Chapter 4. Installation and configuration 21 Procedure Comments in config.properties provide guidance for configuring each property. The config.properties file is in the conf directory of CS_HOME. Configuring access to Engage Use the sp.properties file to define the integration user and the servers that manage data transfer between IBM Campaign and Engage. Procedure Comments in sp.properties provide guidance for configuring each property. The sp.properties file is in the conf directory of CS_HOME. Related concepts: “Security in IBM Engage” on page 8 Proxy server connections You can configure the integration to support communication between Campaign and Engage through a proxy server. If your corporate data security plans demand separation between your corporate data and the public Internet, you can connect to IBM Engage through an HTTP proxy or a FTP proxy. The sp.properties configuration file includes optional configuration properties sets of configurations that you can populate to route traffic through HTTP or FTP proxy servers. To use a web proxy server, you must provide the proxy server address. For authenticated access to the proxy, you must also provide the required user name and password. If authenticated access to the proxy server is required, the user name and password for the proxy are usually assigned by your network administrator or IT staff. You can configure each proxy server independently. If you use the same proxy server for HTTP and FTP, enter the same values for both sets of configuration properties. If you use a single proxy server but do not enter values for FTP configurations, the server passes only HTTP traffic, not FTP traffic. The sp.properties file is in the conf directory of <CS_HOME>. The proxy server credentials are different from the credentials for the integration user. The following diagram illustrates where the proxy server fits in the connection between Campaign and Engage. The following diagram illustrates where the proxy server fits in the connection between Campaign and Engage. 22 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Connecting through an HTTP proxy To route traffic through an HTTP proxy, enter values for the sp.proxy configurations. About this task To upload segmentation data and download contact and tracking data through an HTTP proxy server, enter a URL for the proxy server. For example: sp.proxy.url=http://myproxyserver.example.com:7777. If you do not specify a URL, the system does not route upload and download traffic through the proxy server. If you enter a proxy server URL but do not enter a proxy server user name and password, the proxy server processes upload and download traffic without requiring authentication. Procedure 1. In the conf directory, open sp.properties in a text editor. 2. In the Optional Properties section, modify the following properties. Comments in the file provide guidance for updating each property. v sp.proxy.url Specify the URL for the web proxy server. v sp.proxy.username Specify the user name that is used to access the web proxy server. Leave blank if authenticated access is not required. However, if you specify a user name, you must also specify a password and a URL. v sp.proxy.password Specify the password that is used to access the web proxy server. Leave blank if authenticated access is not required. However, if you specify a password, you must also specify a user name and a URL. 3. Save sp.properties and verify the connection. Note: You must restart the Campaign Analytics server for the configuration changes to take effect. Chapter 4. Installation and configuration 23 Connecting through an FTP proxy server To route traffic through an FTP proxy, enter values for the sp.proxy.ftp configuration properties. About this task If you use the same proxy server for HTTP and FTP traffic, use the same values that you entered in the sp.proxy configurations. If you do not specify values for sp.proxy.ftp, the server does not pass FTP traffic. If you enter a proxy server URL but do not enter a proxy server user name and password, the proxy server processes upload and download traffic without requiring authentication. Procedure 1. In the conf directory, open sp.properties in a text editor. 2. In the Optional Properties section, modify the following properties. Comments in the file provide guidance for updating each property. v sp.proxy.ftp.url Specify the URL for the FTP proxy server. v sp.proxy.ftp.username Specify the user name that is used to access the FTP proxy server. Leave blank if authenticated access is not required. However, if you specify a user name, you must also specify a password and a URL. v sp.proxy.ftp.password Specify the password that is used to access the FTP proxy server. Leave blank if authenticated access is not required. However, if you specify a password, you must also specify a user name and a URL. 3. Save sp.properties and verify the connection. Note: You must restart the Campaign Analytics server for the configuration changes to take effect. Encrypting the integration configurations When you install the Campaign integration with Engage, configuration settings, including passwords, are visible in properties files. To encrypt certain settings so that sensitive information is not visible, run the encryptconfig script. Before you begin Run the trackingDownload script before you encrypt the configuration settings. Running trackingDownload verifies system configurations and the connection to Engage and Campaign data sources. The trackingDownload script is in the bin directory of <CS_HOME>. When you run the trackingDownload script, the system indicates in the command console if the run was successful. Look in the console for any reported errors. You must run trackingDownload without errors before you encrypt property files. The encryption is based on a password-based cryptography standard. By default, the encryption uses a password that is built into the integration code. You can 24 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide further secure the configuration settings by defining a different password. You can define a different password for your environment by defining a custom encryption key. Defining a custom encryption key is optional. However, you must decide whether to define a custom encryption key before you encrypt the properties settings. If you choose to create a custom encryption key, you must do so before you run the encryptconfig script. About this task The encryptconfig script is in the bin directory in the folder where you installed the integration files. The script encrypts configuration values in jdbc.properties and sp.properties. The script encrypts only values that include the terms: password, secret, refresh, or access. When you run the encryptConfig script, the script replaces the displayed values for these settings with a string of encrypted characters. File Property jdbc.properties cs.jdbc.password: Password for the Campaign database user sp.properties sp.password: Password for the integration user. sp.client.secret: System generated authentication credential. sp.refresh.token: System-generated value for establishing access to Engage. Encrypting the value of a configuration property does not prevent you from changing the value. However, each time that you change the value, you must run encryptConfig again to encrypt the new value. You can encrypt values in jdbc.properties and sp.properties simultaneously or encrypt each file separately. Procedure Depending on your operating system, run encryptConfig.bat (Windows) or encryptConfig.sh (UNIX or Linux). v To encrypt jdbc.properties and sp.properties simultaneously, run the script as: <CS_HOME>/bin/encryptConfig. Example: <CS_HOME>\bin\encryptconfig.bat v To encrypt the files separately. Run encryptConfig with the -f parameter as: <CS_HOME>/bin/encryptconfig -f <absolute or relative path to CS_HOME>/conf/<properties file>. Example: <CS_HOME>/bin/encryptConfig.sh -f <CS_HOME>/conf/sp.properties The script encrypts values only in the property file that you specify with the -f option. Related tasks: “Downloading tracking data with a script” on page 62 Related reference: “trackingDownload” on page 61 Chapter 4. Installation and configuration 25 Creating a custom encryption key To apply increased local control over the encryption of sensitive configuration values, you can create a custom encryption key. This task is optional, but does provide an added layer of security. About this task When you create a custom encryption key, you change the password that the encryption script uses to hide sensitive configuration values. If you decide to create a custom encryption key, you must perform this procedure before you run the encryptConfig script. Define the value for the custom encryption key in a flat file. The file is considered the Encryption Key File and system administrators must restrict access to it. Enter the path to this file as a setting in the setenv file. Note: If you change the value of the Encryption Key File, you must remove the current encryption, repeat this procedure to create a new custom encryption key, and run encryptConfig again. If you do not repeat all of the steps in this process, the integration scripts will fail. Procedure 1. In a text editor, create a strong password for the encryption script and save the file as a text file. The file that you create is the Encryption Key File. Save the file in a folder outside of the bin directory. Restrict access to the directory in which you save the file. 2. In the bin directory of <CS_HOME>, edit the setenv file to specify the path to the Encryption Key File. Modify the ENCRYPTION_KEY_FILE setting, as follows. Windows (setenv.bat): set ENCRYPTION_KEY_FILE="Dcom.ibm.emm.integration.silverpop.EncryptionKeyFile=<path to file>\<EncryptionKeyFile>" UNIX or Linux (setenv.sh): ENCRYPTION_KEY_FILE=Dcom.ibm.emm.integration.silverpop.EncryptionKeyFile=<path to file>/<EncryptionKeyFile> Results The new custom password is used to encrypt values in jdbc.properties and sp.properties. What to do next Run the encryptConfig script to encrypt the configuration settings. Removing encryption from configuration properties Some situations require that you remove the encryption from encrypted properties. For example, to change the value for the Encryption Key File, you must remove encryption from all currently encrypted values before you can proceed. About this task Encrypted values can be found in jdbc.properties and sp.properties. The encrypted values appear as random strings. 26 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Procedure 1. In jdbc.properties and sp.properties, locate the encrypted values. Replace each encrypted value with its correct unencrypted value. 2. Save the file. On Windows, if you were using a custom encryption key and then stopped using the key, close all command prompts. Results After you complete this task, the configuration values appear without encryption. UBX Toolkit installation To download tracking data for SMS messages and mobile push notifications, you must use the UBX Toolkit. The UBX Toolkit requires that you install scripts on a local server and that you establish a UBX user account. For information and instructions about how to install and configure the UBX Toolkit, including how to establish a UBX user account, see the documentation that describes the UBX Toolkit on IBM Knowledge Center: http://www.ibm.com/ support/knowledgecenter/SS9JVY/UBXtoolkit/Installation_toolkit/ UBX_Toolkit_installation_and_configuration.dita. Preparing UBX endpoints for the integration The integration supports the use of the UBX Toolkit to download tracking data for email, SMS, and mobile push. The UBX Toolkit requires IBM Universal Behavior Exchange (UBX). You must register Campaign and Engage as UBX endpoints and subscribe to the events that Engage can provide for the various channels. About this task In UBX, interactions between individuals and a business solution, such as Engage, are referred to as events. Recipient responses to email, SMS, and mobile app messages are considered events. UBX provides a way for business solutions that observe an event to send attributes that describe the event to other business solutions. The destination solution can use the event data to create a specific business outcome or perform some sort of analysis. Business solutions that provide event data or receive event data are referred to as UBX endpoints. In terms of the integration between Campaign and Engage, Engage is an event publisher (producer) endpoint, and Campaign is an event subscriber (consumer) endpoint. To establish the required relationships between Campaign and Engage through UBX, you must register Campaign and Engage as UBX endpoints. The UBX Toolkit provides a script and related configurations that you can use to perform the required registration steps. After you register Campaign and Engage as endpoints, you must log in to UBX and specify the events that Engage can send to Campaign. This process is referred to as subscribing to events. Procedure 1. In the UBX Toolkit, register Campaign as an event consumer endpoint. Chapter 4. Installation and configuration 27 For information about how to use the UBX Toolkit to perform the registration, see the instructions for registering an event destination, available in IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBXtoolkit/ EventConsumer_toolkit/Registering_as_an_event_destination.dita. 2. In UBX, register Engage and Xtify as event producer endpoints. a. On the Endpoints tab, click Register new endpoint. b. In the endpoint registration wizard, select Engage as an event producer endpoint. Follow the on-screen instructions for how to complete the registration. c. Repeat steps a and b for Xtify. During the registration process, you are asked to provide the refresh token that Engage provided for your organization and for the name of the pod that supports your Engage organization. For more information, consult the description of UBX endpoint registration that is available in IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/ Endpoints_ubx/Endpoint_registration_ch.dita. 3. In UBX, subscribe to events from Engage. a. On the Events tab, click Subscribe to events. b. Under IBM, select Engage and select all available email and SMS events. c. Under IBM, select Xtify and select all available push notification events. d. Select a destination. Under IBM, select Campaign as the event destination. e. Click Subscribe. For more information, consult the documentation for event publication and subscription that is available in IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/ Events_ubx/Event_sharing.dita. What to do next After events are subscribed and endpoint is configured, Campaign downloads event data into a system table. If another event is subscribed, it is added to single table named UA_GEN_EVENT_RECORD. You can configure the schedule and frequency of an event download. Add the following configuration to provide a UBX URL: Affinium|Campaign|partitions|partition1|Engage|ubx. 28 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 5. Data upload - Campaign to Engage The integration package includes scripts to upload contact data and segmentation from Campaign to Engage. Add these scripts to Campaign flowcharts to automate the process. Always ensure that the field names and formats in Campaign match the names and formats in Engage. Use the scripts that are provided with the integration to upload the data that is required to compose, transmit, and track personalized messages in Engage. You can automate communication with your customers by configuring data uploads that trigger Engage mailings after uploads from Campaign. The scripts can direct the uploaded data to Engage databases, contact lists, and relational tables. You can run the scripts manually from a command line or run them automatically when you run a Campaign flowchart. Uploading data from Campaign to Engage involves the following general steps. v Determine what to upload. Examine the data that is available through Campaign. The data that you upload must contain at least one field that can be used to identify specific individuals. See “What to upload from Campaign” on page 30 v Decide where to upload the data in Engage. You can upload the data to Engage contact lists, databases, and relational tables. At this point, you can upload data by running a script that is included as part of the integration. However, you can proceed to configure a flowchart to automate the upload. See “Where to put uploaded data in Engage” on page 30 and the IBM Marketing Cloud Knowledge Base at https://portal.silverpop.com. IBM Silverpop Knowledge Base at https://portal.silverpop.com. v Create a placeholder offer. The integration does not support offers, but you must configure an empty offer so that you can add a Mail List process to the flowchart. See “Placeholder offer for the Mail List process” on page 42. v Define a flowchart trigger. The trigger includes an upload script. See “Creating a flowchart trigger for automatic data upload” on page 40 v Configure a flowchart that includes a Mail List process. The output of the Mail List process is a file that you can upload to Engage that contains the contact data that you want to upload. See “Flowchart design for data upload to Engage” on page 39. v Run the flowchart to upload the data. Depending on how you configure the upload script, the upload can trigger a preconfigured mailing, SMS program, or push notification by Engage. You can run the flowchart manually or schedule the flowchart to run at a future date, or run recurrently. Each scheduled or recurrent run of the flowchart uploads data that is available at the time. See “Uploading data by running a flowchart” on page 38 or “Uploading data by running a script” on page 38. © Copyright IBM Corp. 2014, 2017 29 What to upload from Campaign You can upload data segmentation and contact data that you define in Campaign to databases, contact lists, and relational tables that you define in Engage. Upload only the data that you require to compose and send personalized messages to your target audience. The data that you upload from Campaign typically includes the following types of segmentation and contact data. v Any type of data that can be used to identify a specific individual. For example, and account ID or email address. v Information that is required to direct a message to an individual. For example, email address, SMS number, or mobile phone number. v SMS consent data. You cannot send an SMS message to an individual unless you have the consent of the individual in the form of an Opt-In for each SMS program. v Any data that you can use to personalize the messages that you send. To upload segmentation data from Campaign customer files to an Engage database or contact list, use the contactUpload script. To upload Campaign dimension table data to an Engage relational database, use the tableUpload script. You can run the contactUpload or tableUpload script manually from the command prompt. You can also configure Campaign flowcharts to run either script as a flowchart trigger to upload the flowchart output automatically to Engage. Both scripts upload Campaign data in a tab-separated file. The first line in the tab-separated file must contain the field names for the Campaign data. The Campaign field names must match column names in the Engage database. If the column names in Campaign do not match, you have several options to either rename the fields or map them to the corresponding fields in Engage. The preferred approach is to rename the Campaign fields as you map them to various flowchart processes. Other options include defining derived fields or custom mapping files, if necessary. Related concepts: “Data management for Campaign and Engage” on page 8 “Where to put uploaded data in Engage” “Data required for SMS consent” on page 78 Related reference: “contactUpload” on page 33 “tableUpload” on page 36 Where to put uploaded data in Engage In Engage, you can define several ways to store and segment data that you upload from Campaign. The destination for the data upload determines which upload script to run and how to configure the script. You can upload data from Campaign to any of the following types of data collections in Engage. 30 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Databases Engage provides Single Opt-in and Double Opt-in databases. They contain information about contacts that you can use to address and personalize messages that you send with Engage. Although you can upload contact information to a double-opt in database, the new records are not marked as opt-in until the message recipients complete the double opt-in process. You specify the database ID in an upload script. Contact lists You can define one or more contact lists within a single Engage database. Each contact list defines a subgroup of message recipients and data that is related to them. You can upload data from Campaign directly to a contact list. When you upload data to a contact list, you are also adding the data to the parent Engage database. You specify the contact list ID or name in an upload script. If the contact list that you specify does not already exist in Engage, the upload script creates a new contact list. Relational tables An Engage relational table can associate multiple lines of data with a single database record. For example, in a relational table you can map records for multiple purchases, event attendances, or rewards to each message contact that is represented in the table. Typically, you upload contact data and segmentation from Campaign dimension tables to Engage relational tables. You specify the table ID in an upload script. Related concepts: “Data management for Campaign and Engage” on page 8 “What to upload from Campaign” on page 30 “Requirement to match Campaign fields to Engage fields” on page 43 “Requirement to match date formats” on page 46 Finding contact list, database, and relational table IDs in Engage Engage assigns each database, contact list, and relational table a unique ID. You can specify this ID when you upload data from Campaign. For contact lists only, you can also specify the contact list name. About this task You can find the ID for a database, contact list, or relational table in Engage. The ID is described in the additional details for each type of data collection. Chapter 5. Data upload - Campaign to Engage 31 Procedure 1. On the Data tab in Engage, navigate to the database, contact list, or relational table that you want to specify. The names display as links on the Databases, Contact Lists, or Relational Tables tabs. 2. Click the name of the database, list, or table that you want to identify. 3. Click Show Additional Details. The ID value displays on the right side of the summary window. Related reference: “contactUpload” on page 33 “tableUpload” on page 36 “contactDownload” on page 50 “trackingDownload” on page 61 Finding mailing IDs in Engage When you configure the contactUpload script to trigger a mailing in Engage, you can specify the mailing ID for the mailing that you want to send. You must access Engage to determine the appropriate mailing ID. Procedure 1. On the Content tab in Engage, got to View Mailings and click Templates. The Mailings and Templates page opens and displays a list of available mailings as links. 2. In the list of mailings, hold the cursor over a mailing name. The mailing ID displays in a tooltip. Related reference: “contactUpload” on page 33 “tableUpload” on page 36 “contactDownload” on page 50 “trackingDownload” on page 61 How to upload data to Engage Uploading data from Campaign to Engage requires configuring and running Campaign flowcharts and command-line scripts. You must also have access to the Engage account to which you upload the data from Campaign. To upload contact data from Campaign to Engage, you must run a Campaign flowchart to select the data. The flowchart must contain a Mail List process to assemble the data as a tab-separated file (.tsv) that serves as the input to a data upload script. You can automate the selection and upload process by adding the script to the Mail List process as a flowchart trigger. To upload segmentation data from Campaign customer files to an Engage database or contact list, use the contactUpload script. To upload Campaign dimension table data to an Engage relational database, use the tableUpload script. Both scripts upload Campaign data in a tab-separated file. 32 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide You can run the scripts manually from the command prompt or run them programmatically by incorporating them into Campaign flowchart triggers. The script parameters determine how and where the Campaign data is uploaded to Engage. The first line in the tab-separated file must contain the field names for the Campaign data. The Campaign field names must match column names in the Engage database. If the column names in Campaign do not match, you have several options to either rename the fields or map them to the corresponding fields in Engage. The preferred approach is to rename the Campaign fields as you map them to various flowchart processes. Other options include defining derived fields or custom mapping files, if necessary. contactUpload Use contactUpload.bat (Windows) or contactUpload.sh (UNIX or Linux) to upload a tab-separated file that contains contact and segmentation data from IBM Campaign to databases and contact lists in IBM Engage. The contactUpload script is located in the bin directory of the <CS_HOME> directory. The <CS_HOME> directory is the folder on the Campaign Analytics server where the integration files are installed in ../campaign/partition/partition[n]/<CS_HOME> Confirm that the column names that you select in Campaign match the corresponding column names in the Engage database. If you cannot match the names, you must use other methods, such as using derived fields or a custom mapping file to establish the match. If you use a custom mapping file, you must reference it as a parameter in the contactUpload script. You can run the contactUpload script from a command line on the Campaign Analytics server or include the script in a flowchart trigger. You must have access to the Campaign Analytics server and user permissions that allow you to run scripts. You can specify an Engage contact list by its list ID or by its name. Contact list names must conform to Engage contact list name requirements and must contain only ASCII characters. If the list name includes spaces, you must enclose the name in quotation marks. You can also create a new contact list by specifying a contact list name that is not already in use by your organization in Engage. Usage contactUpload (-i <inputFile>) (-l <datbaseID> [-p <contactListIDs> -t <import_type>] | -m <mappingFile>) [-e <templateID> -g <recipientListID> -r -c <configPropertiesFile> -s <spPropertiesFile> -j <jdbcPropertiesFile> -k <campaigncode> -u <channel> -b <SMS program ID>] Option Description -i | --inputFile (Required) Path to the tab-separated file that contains the list of values to be uploaded to Engage. Absolute or relative path to <CS_HOME>. -l | --listID ID of the Engage database that receives the uploaded values. Specify the ID as defined in the Engage interface. Do not use with custom mapping file (-m). Does not upload values to child databases. Chapter 5. Data upload - Campaign to Engage 33 Option Description -m | --mapfile Absolute or relative path to a file (.xml) that defines the mapping between fields in the input file and fields in the Engage database. Specify the database ID in the mapping file. Optionally, specify contact list ID values. Do not use with: (-l), (-t), (-p). -t | --ImportType If no value is specified, the system adds and updates records in the specified Engage database with data from the input file (-i). All input values are considered Opt-In contacts. If you specify (-t OPT_OUT), the uploaded records that match existing records are added as Opt-Out contacts. Do not use if you specify a custom mapping file (-m). -p | --contactListIDs Use to upload contact data to one or more contact lists that are defined in Engage. Specify the ID or name of the contact list to which you are uploading. Upload to multiple lists by providing a comma-separated list (no spaces) of contact list IDs or names. If you pass one or more contact list names, the script gets a list of all shared contact lists for your organization from Engage and matches to existing list IDs. If the script finds a match to an existing contact list, the script uploads to the list. If the script does not find an existing list that matches the list name that you pass, the script calls an API to create a new shared contact list in Engage with the name that you passed in the script. Contact list names cannot start with a digit, must contain only ASCII characters, and must conform to Engage list naming requirements. List names that include spaces must be enclosed by quotation marks. Do not use this option if you specify a custom mapping file (-m). -e | --MailingTemplateID When specified, Engage sends an email, SMS, or mobile app messages after the segmentation data is uploaded and added to the database that you specify with the (-l) option. Enter the ID of the template that defines the message configuration. v For email: the ID of the email mailing template v For SMS and push: the published ID of the SMS or mobile app message The template must specify the same contact list as you specify with the (-p) option or the (-m) option. When uploading data to multiple contact lists, use the (-g) option to identify the list of message recipients. See also the (-k) option to prepend a campaign code to the mailing name. -g | --RecipientListID Use when uploading values to multiple contact lists and sending an email (-e). Specify the ID or name of the contact list that defines the single list of message recipients. If you pass one or more contact list names, the script gets a list of all shared contact lists for your organization from Engage and matches to existing list IDs. If the script finds a match to an existing contact list, the script uploads to the list. With the (-g) option, if the script does not find an existing recipient list that matches the list name that you pass, the script fails and reports an error. Not supported for SMS messaging. -r | --removeContactData 34 When specified, the system removes values in a contact list and replaces them with the values that are contained in the uploaded tab-separated file. You must specify the contact list IDs with either the (-p) option or in a mapping file (-m). IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Option Description -c | --configPropertiesFile Direct the integration to use alternate configuration properties. By default, the integration system configurations are contained in a file named config.properties in the <CS_HOME>/conf directory. However, if your business case requires alternate configurations, specify this property to direct the integration to use configurations that are contained in a config.properties file in a different location or with a different name (the file extension must remain the same). Paths with spaces must be enclosed in quotation marks. -j | --jdbcPropertiesFile Direct the integration to use an alternate database. By default, the integration uses the database that is specified in jdbc.properties in the <CS_HOME>/conf directory. However, if your business case requires an alternate database, specify this property to direct the integration use the database that is specified in a jdbc.properties file in a different location or with a different name (the file extension must remain the same). Paths with spaces must be enclosed in quotation marks. -s | --spPropertiesFile -k | --campaignCode Path to the sp.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/sp.properties.] Paths with spaces must be enclosed in quotation marks. When used with the (-e) option, you can add information to the name of an Engage mailing as a prefix. You can use this option to distinguish between mailings if you upload data to send multiple mailings. For example, you can add a campaign code that you define in Campaign to the mailing name. However, the additional identifier can be any string that you define. The additional information that you add is preserved as part of the mailing name during data download. The mailing name that you define is available in the MailingName field in the integration tracking tables. The format of the mailing name is <campaigncode> <mailing template name> <timestamp>. -u | --channel Specify the messaging channel. You can specify only one channel. Defaults to email if you do not specify a value, Valid values: Email messages: email SMS messages: sms Mobile push notifications: push -b | --smsprogid Specify the SMS program ID, as defined in Engage. Required for uploading SMS consent data. -f | --syncfields Specify the name of the field, or comma-separated list of fields, that provides the identifier value to be used to distinguish between individual message recipients. Overrides system-level configuration, if such a configuration is defined. Required for upload of SMS consent data. For SMS, the value for the SYNC field must be the mobile phone number. -n | --sto Enable Send Time Optimization (STO). Email only. The Engage database that you specify with -l | --list ID must be enabled for STO. Chapter 5. Data upload - Campaign to Engage 35 Option Description -x | --consentOverrideonNoChange Allow the system to refresh an individual's consent status for a particular SMS program. Valid values: true | false Default: true true: the system can refresh the consent status, including consent date, when the record is updated. false: the system does not refresh consent status during record updates. Consent date is preserved. -y |-consentHonorOptOutStatus Control the ability of Engage to override recipient consent status for a specific SMS program. Valid values: true | false Default: true true: Honor the recorded consent status. false: Override the recorded consent status. Note: Overriding an Opt-Out consent might violate relevant regulations. Modify the default only if applicable regulations allow. Related concepts: “What to upload from Campaign” on page 30 “CS_HOME directory” on page 18 “Additional identifiers in mailing names” on page 43 “SMS Opt-in and Opt-out synchronization between Campaign and Engage” on page 82 Related tasks: “Finding contact list, database, and relational table IDs in Engage” on page 31 “Finding mailing IDs in Engage” on page 32 “Uploading data by running a flowchart” on page 38 tableUpload Use tableUpload.bat (Windows) or tableUpload.sh (UNIX or Linux) to upload a tab-separated file that contains contact and segmentation data from IBM Campaign to a relational table in IBM Engage. You must create the Engage relational table to which you are uploading data before you run the script to upload data from Campaign. Note: The tableUpload script does not accept table names as input, nor does it create a new table if the table does not exist in Engage. The tableUpload script is not channel-specific. You can run the tableUpload script to upload data for email, SMS, and mobile app messaging. Relational tables can accept data from Campaign dimension tables or data tables that you create in Campaign. Relational tables are useful for storing multiple records that are related to a single record. For example, for each individual that is represented in the Campaign table, you can store the list of purchases for the past year or the list of preferred travel destinations. 36 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Confirm that the column names that you select in Campaign match the corresponding column names in the Engage database. You can change column names when you map tables to a flowchart. Use the ability to change column names in Campaign to match the names in Engage. If you cannot match the names, you must use other methods, such as derived fields or custom mapping files to establish the match. If you use a custom mapping file, you must reference it as a parameter in the tableUpload script. The tableUpload script is in the bin directory of the <CS_HOME> directory. The <CS_HOME> directory is the folder on the Campaign Analytics server where the integration files are installed in ../campaign/partition/partition[n]/<CS_HOME> Run the tableUpload script from a command line on the Campaign Analytics server. You must have user access to the Campaign Analytics server and permissions to run scripts. Usage tableUpload (-i <inputFile>) (-l <tableID> | -m <mappingFile>) [-c <configPropertiesFile> -s <spPropertiesFile> -j <jdbcPropertiesFile>] Option Description -i | --inputFile (Required) Path to the tab-separated file that contains the list of values to be uploaded to Engage. Absolute or relative path to <CS_HOME>. -m | --mapfile Absolute or relative path to a file (.xml) that defines the mapping between fields in the input file and fields in the Engage database. Specify the database ID in the mapping file. Optionally, specify contact list ID values. Do not use with: (-l), (-t), (-p). ID of the Engage relational table that receives the uploaded values. Specify the ID as defined in the Engage interface. -l | --listID Do not use with custom mapping file (-m). -c | --configPropertiesFile Path to the config.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/configuration.properties.] Paths with spaces must be enclosed in quotation marks. -s | --spPropertiesFile Path to the sp.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/sp.properties.] Paths with spaces must be enclosed in quotation marks. -j | --jdbcPropertiesFile Path to the jdbc.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/jdbc.properties. Paths with spaces must be enclosed in quotation marks. Related concepts: “What to upload from Campaign” on page 30 “CS_HOME directory” on page 18 Related tasks: “Finding contact list, database, and relational table IDs in Engage” on page 31 “Finding mailing IDs in Engage” on page 32 Chapter 5. Data upload - Campaign to Engage 37 Upload preparation and requirements Uploading data from Campaign to Engage requires certain preparations and imposes specific requirements. Before you upload contact data and segmentation to Engage, confirm that you can meet the requirements and are ready you proceed. To upload contact data and segmentation from Campaign, ensure that the database field names in Campaign match the corresponding field names in Engage. You have several options to make certain that the field names match. When you upload date information from Campaign to Engage, the two applications must format the date information in the same way. Configuring a Campaign flowchart to upload contact data in a form that Engage can accept requires that you add a Mail List process to the flowchart. The Mail List process requires a Campaign offer. Although the Campaign integration with Engage does not support Campaign offers, you must define an empty placeholder offer to satisfy the requirements of the Mail List process. Uploading data by running a script To manually upload data from Campaign to Engage, run the contactUpload or tableUpload script from a command line on the Campaign Analytics server. Before you begin Run a Campaign flowchart that selects address and other data to describe message recipients. The flowchart must contain a Mail List process so that the output of the flowchart run is a tab-separated file that you can upload. About this task Running the contactUpload script manually can be useful during testing and troubleshooting before you automate data uploads by adding the script to triggers in Campaign flowcharts. Procedure 1. Open a command line. 2. Change directory to the bin directory in your <CS_HOME> directory. 3. Run either contactUpload or tableUpload with the options that you require. Help is available for the scripts on the command line. Enter -h or --help. Uploading data by running a flowchart Automatically uploading Campaign data to an Engage database involves a series of procedures that involve a flowchart that contains a Mail List process and a flowchart trigger. Running the flowchart creates a tab-separated list of field values that is uploaded to Engage automatically when the flowchart run finishes. Before you begin If you plan to send messages in Engage automatically after the flowchart run, you must configure the mailing, SMS program, or push notification before you run the flowchart. 38 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide About this task Using a Campaign flowchart and a stored trigger to automatically upload data from Campaign to Engage involves the general steps that are listed in the following table. The trigger that you add to the Mail List process contains the contactUpload script. The trigger configuration determines what you upload to Engage. Procedure 1. In Campaign, create or edit a flowchart. Modify the flowchart as needed to select the information that you want to upload to Engage. Match Campaign field names to Engage field names. See the IBM Campaign User's Guide for information about creating and configuring flowcharts. See also, “Flowchart design for data upload to Engage” 2. Create a flowchart trigger that references an upload script and the tab-separated file that is generated by the Mail List process. For more information, see “Creating a flowchart trigger for automatic data upload” on page 40 3. Add a Mail List process to the flowchart. The Mail List process must be the final process in the flowchart. Connect upstream processes as inputs to the Mail List process. 4. Configure the Mail List process. Add a trigger to define what to upload. For more information, see “Configuring a Mail List process for automatic data upload to Engage” on page 41 5. Save and run the flowchart. When the run finishes, the upload script triggers an upload of the flowchart output to the specified destination in Engage. For more information about the destination of the data in Engage, see “Where to put uploaded data in Engage” on page 30. What to do next Log in to IBM Engage to compose and send messages based on the data that you uploaded. After you send the messages, download contact and tracking data to evaluate the response to your message and plan the next steps. Related reference: “contactUpload” on page 33 Flowchart design for data upload to Engage The flowcharts that you create to select data for upload to Engage use the same flowchart processes and follow the same design practices as flowcharts that you create for other purposes. However, several additional considerations apply when you design a flowchart to upload data to Engage. v The names of the database fields that you select with the Campaign flowchart must exactly match corresponding fields in Engage. You must be familiar with Chapter 5. Data upload - Campaign to Engage 39 the Engage databases and tables that you want to populate. As necessary, assign Campaign Field Names that match the field names in Engage. v Flowcharts that you use to upload data to Engage must contain a Mail List process that generates output as a tab-separated file. Add the Mail List as the final process in the flowchart. v Because uploading data is often an automated process, flowcharts that you use to upload data to Engage contain a trigger to run an upload script as soon as a flowchart run completes. Related tasks: “Creating a flowchart trigger for automatic data upload” “Configuring a Mail List process for automatic data upload to Engage” on page 41 Creating a flowchart trigger for automatic data upload In a Campaign flowchart, you can create a stored trigger that incorporates an upload script. You can configure the script to trigger a data upload to an Engage database, contact list, or relational table each time the flowchart run finishes. Before you begin Confirm that you have permissions to create triggers in Campaign. Check with your Campaign administrator. Procedure 1. When you edit a flowchart, open the Options icon Triggers. and select Stored The Stored Trigger Definitions window opens. 2. Click New Item. The data fields for the new trigger appear on the right of the window. 3. You can select a folder to save the trigger to in the Save Under list. Note: The folder location governs which users can access the trigger, based on the folder's security policy. 4. Enter a name for the trigger in the Name field. v You cannot use spaces in the string, but you can use underscores ( _ ). v This name must be unique within the folder where you save it. 5. If you are creating a trigger in the top-level folder, select a security policy, or keep the default. 6. You can enter a description of the trigger in the Note field. You can provide a free-form text description of the trigger for documentation purposes. You also might want to keep a modification history of who modified the trigger, when, and what changes were made. 7. In the Command field, enter the contactUpload or tableUpload script. The choice of script and how you configure the script parameters determine whether the flowchart output uploads to an Engage database, one or more contact lists, or to a relational table. 40 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Note the name and location of the input file that you specify with the -i option. You must specify the same file as the output of a Mail List process when you add the trigger to a flowchart to automatically upload data to Engage. For example, to upload to an Engage database in Windows, you might enter: <CS_HOME>\bin\contactUpload.bat -i C:\IBM\Campaign\partitions\ partition1\dataexport\SPFlowchart1.tsv -l 98765 To upload to a relational table in UNIX or Linux: <CS_HOME>/bin/ tableUpload.sh -i /data/IBM/Campaign/partitions/partition1/dataexport/ SPFlowchart1.tsv -l 56781 <CS_HOME> represents the directory on the Campaign Analytics server where you installed the Campaign integration with Engage files. 8. Click Save and Close. What to do next Configure a Mail List process to automatically upload contact data to Engage. Related concepts: “Flowchart design for data upload to Engage” on page 39 Configuring a Mail List process for automatic data upload to Engage In a Campaign flowchart, you can add a Mail List process to assemble the output of upstream flowchart processes into a tab-delimited file that can be uploaded to Engage. To support automatic data upload to Engage, the Mail List process works with a stored flowchart trigger. About this task In a flowchart, you can connect one or more configured processes to the Mail List process. The processes that you connect must produce output cells, which serve as input to the Mail List process. For example, a Select process produces a list of IDs, so its output can serve as input to the Mail List process. Procedure 1. Open a Campaign flowchart for editing. Either add a Mail List process to the flowchart or configure an existing Mail List process. 2. Connect one or more configured processes as input to the Mail List process. Select input cells that, when you run the flowchart, provide the data that you want to upload to Engage. Important: All of the cells that you select as input cells must have the same audience level. 3. Double-click the Mail List process in the flowchart workspace. The process configuration dialog opens. 4. Use the Fulfillment tab to specify the input to the list and to specify the form of the output. a. From the Input list, specify the cells to use as the data source for the list. Note: The Multiple Cells option is available only if the input process generates multiple cells or if more processes feed into the contact process. Chapter 5. Data upload - Campaign to Engage 41 b. By default, Enable Export To is selected by default. Leave Enable Export To selected, and configure the following options: v Select File from the Enable Export To list. Provide a file name and other details, including where to save the file. Make a note of the file name and location. v Specify the output file as a Delimited File and select Tab as the delimiter. Select Include Labels in Top Row. v Select Replace All Records to indicate how to handle updates to the output file. This option removes any existing data from the file and replaces it with the new information. c. Select Send Trigger(s), and choose the trigger that you want to send. The selected trigger is listed in the Send Trigger(s) field. Note: Always place a question mark (?) after the trigger command so that, if the trigger fails, a red X displays on the Mail List process to inform you of the problem. 5. On the Treatment tab, select the placeholder offer. 6. On the Personalization tab, specify which fields to add to the upload file. For example, add a field that provides an email address. v The Export Field list indicates which fields to write to the output list. v If you selected a file on the Fulfillment tab, the Export Field list is empty and you must specify which fields to output. v When you select Candidate Fields, you can click the arrow next to an item to expand it. Use Ctrl+Click or Shift+Click to select multiple fields. v To view the values in a field, select the field and click Profile. v Use the Add and Remove controls to adjust the contents of the list. v The order of the fields in the Export Fields list determines the order that the data is written out. 7. On the Log tab, clear Log to Contact History Tables. You must have the appropriate permissions to enable or disable the contact history log options. 8. (Optional) Use the General tab to assign a name and descriptive notes to the process. 9. Click OK. Related concepts: “Flowchart design for data upload to Engage” on page 39 Placeholder offer for the Mail List process The Campaign integration with Engage uses a Mail List process to create the tab-separated file that the upload scripts upload to Engage. The design of the Mail List process requires a reference to a Campaign offer. To meet this requirement, you must create a placeholder offer that does not display an offer to message recipients. The placeholder offer is an empty offer. You can turn contact history off for the offer because it does not appear to message recipients. Note: The standard Campaign integration with Engage does not support Campaign offers. 42 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide To create the placeholder offer, you must log in to Campaign with a user role that has permissions to Add Offers. When you create the offer, you must provide an offer name. For easier offer management, create a name that clearly relates the offer to the Campaign integration with Engage. For example, CampaignEngageOffer. The offer is automatically associated with a Campaign security policy and Campaign assigns a system-generated offer code. For more information about creating offers, see the IBM Campaign User's Guide. Additional identifiers in mailing names The contactUpload script provides an option to add additional information to the names of mailings that you trigger automatically after a data upload. You can use the identifiers for tracking and retargeting message responses. You can use the contactUpload script to automatically trigger an Engage mailing when you upload data to an Engage contact list. The mailing is based on an existing Engage mailing template that assigns a mailing name. The contactUpload script provides an option to add additional information to the mailing name when you trigger a mailing after a data upload. For example, you can add a campaign code that you define in Campaign to the name of the mailing that Engage sends. You can use this additional information to distinguish between multiple mailings. The mailing name, including the additional identifier, is included as part of response data downloads. The mailing name is added to the MailingName field in the integration tracking tables. After downloading the data, including the mailing name, you can run additional processes to segment the response data based on the additional identifier that you added to the mailing name. Related reference: “contactUpload” on page 33 Requirement to match Campaign fields to Engage fields You must match or map the field names in Campaign to the corresponding field names in Engage. The Campaign integration with Engage supports several methods to match or map field names in the uploaded data. The integration data upload scripts define the field names in the upload .tsv file based on the field names present in the Engage database, contact list, or relational table that you specify as a script parameter. The field names in Campaign and Engage must match exactly. If the field names do not match, you must either change the field names or define a mapping file that specifies the field relationships. Use the following methods, listed in the table in order of simplicity, to match the field names in Campaign and Engage. Chapter 5. Data upload - Campaign to Engage 43 Table 2. Methods to match fields Matching method More information Change field names in Campaign. “Mapping field names in a flowchart” As you add flowchart processes to a Campaign flowchart, define Campaign Field Names that match the field names in Engage. As necessary, match all field names before you configure the Mail List process to create the tab-separated upload file. The first row of the upload file defines the column names for the data that you upload. Use derived fields in Campaign. If you cannot rename fields as you map tables to a flowchart, you can define derived fields that can map the names. For example, derived fields provide a means to resolve differences in date formats when you upload dates to an Engage relational table. Because you assign a name to the derived field, you can match a name that is defined in Engage. Create a custom mapping file. In cases where renaming fields or creating derived fields are not sufficient, you can configure a custom mapping file. The integration download package contains examples of custom mapping files that you can use as a model. See the IBM Campaign User's Guide for more information about creating derived fields. “Mapping fields with a custom XML file” on page 45 Related concepts: “Where to put uploaded data in Engage” on page 30 Mapping field names in a flowchart When you map customer tables to a flowchart, you can change the name that Campaign uses to identify the fields in the table. You must match the field name that Campaign uses for a field to match the field name that is defined in the Engage database. About this task Campaign provides a series of related screens to map new tables and remap existing tables. When you specify a table name and field information, you can change the IBM Campaign Field Name to match a corresponding field in an Engage database. Changing the name of the field in the flowchart configuration is the simplest way to match field names in Campaign to field names in Engage. Changing the name in the table mapping does not change the field name in your customer database. Procedure 1. Edit the flowchart that you are using to select the data to upload to Engage. 2. From the Admin menu in the flowchart toolbar, select Tables. 3. Specify a table. Do either of the following steps. 44 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide a. Select an existing table that contains the data that you want to map and remap the table. b. Add a table and select a table type Click Next. 4. Select Map to Existing Table in Selected Database, select the data source and click Next. 5. Select a source table. The source fields in the table that you select are mapped automatically to fields in the base record table that you are creating. To change the automatic mappings, use the field manipulation controls to add and move fields to the new table. Click Next. 6. In the IBM Campaign Field Name column, click the name of the field that you want to rename. Enter a name that exactly matches the corresponding field name in Engage. 7. Configure audience levels, if necessary and continue to click Next until you advance to the last configuration screen. 8. Click Finish. Mapping fields with a custom XML file In cases where changing the Campaign field name in the flowchart configuration is not feasible, you can create custom mapping files that explicitly define the relationship between Campaign fields and fields in Engage. About this task In the mapping file, define how the fields in the Campaign database provide data that populates corresponding fields in the Engage non-keyed database. A mapping file provides column names that are indexed to the sequence of values as they appear in the tab-separated list that is uploaded to Campaign. The first row of the tab-separated list provides column names. The mapping file defines the column names in the same order. The mapping file also provides information about how to process the list of values. Mapping files can become outdated due to changes to the database by multiple users over time. Changes in the configuration of the Mail List process that is used to upload data can also cause the mapping file to be inaccurate. Before you use a mapping file, confirm that the field mapping is still correct. A mapping file is a parameter to contactUpload and tableUpload. You can assign any name to this file. The parameters that are given to the scripts specify the location and name of the mapping file. The Campaign integration with Engage provides two example mapping files that you can use as models to create a custom mapping file. v example_mappingFile.xml – example for upload to databases and contact lists v example_SMSmappingFile.xml – example for upload to support SMS messaging The SMS mapping file contains additional fields so that you can provide values for recipient consent. Using a mapping file is required when you use contactUpload and tableUpload to provide consent data from Campaign for use in SMS messages that are sent with Engage. When you upload Campaign segmentation data to an Engage database, you must designate a field that provides information that identifies each user. Use the Chapter 5. Data upload - Campaign to Engage 45 <SYNC_FIELDS> property to define the field in Campaign that provides information that identifies each contact. Specify the modified mapping file as a parameter in the upload script. In <SYNC_FIELDS>, the value of the <name> field must be something that is present in the uploaded Campaign data that uniquely identifies each contact. For SMS messaging, the value of the SYNC_FIELD must be the recipient's mobile phone number. Procedure 1. Go to the conf directory in <CS_HOME>. 2. Copy the mapping file that you want to modify. You can rename the example file and save it in <CS_HOME>, or you can save it in another directory. Note the name and location because you must specify the path when you use it in an upload script. 3. In a text or XML editor, open the mapping file. The file contains extensive comments to guide you in creating the custom mapping file for your application. 4. Save your changes. When you configure an upload script, you can use the -m (or --mapFile) option to specify the location of the mapping file. Requirement to match date formats If the Campaign data that you are uploading includes date information, you must take additional steps to preserve the date format. In some cases, you might need to convert the date to a format that Engage supports. Dates can be formatted in many different ways. If Campaign and Engage do not use the same date format, Engage might not correctly interpret the uploaded date information. The system might generate an error or an incorrect date. The Campaign integration with Engage provides the following ways to ensure that date information from Campaign is uploaded correctly to Engage. v You can set a default date format for all data uploads as a configuration property. However, date formats that you define in a custom mapping file override the default date format. v You can define date formats other than the default format as derived fields in the Campaign flowchart that you use to select the data in your customer tables. For example, defining derived fields is the only way correct date formats that do not match when you upload dates to Engage relational tables. Review your customer database to determine the format of date information that you might want to use in Engage. Determine the most common format and configure it as the default date format. Define custom date formats as necessary when you prepare the data upload. Related concepts: “Where to put uploaded data in Engage” on page 30 Default date format for uploads To help ensure that date values upload to Engage in the expected format, you can define a default date format in the config.properties file. 46 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide By default, the config.properties file is in the conf directory where the integration files are saved on the Campaign Analytics server. The default date format is saved as the value for the importlist.dateformat configuration. The default setting for importlist.dateformat is MM/dd/yyyy. For example, a date in this format displays as 06/15/2014. You can change this value to match the date format that is most common in your customer database. The date format that you select as the default format must match one of the date formats that is supported by Engage. Uploading dates with derived fields You can upload dates that do not follow the organization default date format by defining a derived field to upload the date with a different date format. About this task You can define a custom date format with a derived field to pass date information to an Engage relational table. Derived fields are variables that do not exist in a data source and are created from one or more existing fields, even across different data sources. You can use a macro in Campaign to define a derived field that provides date values in a way that Engage can recognize, based on the date format used by a field in your customer database tables. For more information about working with derived fields, see the IBM Campaign User's Guide. Derived fields for uploading dates to Engage must define the following elements. v Campaign field that you want to map. v Date format for the Campaign field. v Output date format. The output date format must match a date format that is supported in a Engage database or relational table. To use a derived field to upload a date field to Engage, you can use the DATE_FORMAT macro in a flowchart process configuration. The macro uses the following syntax. DATE_FORMAT(date_string, input_format, output_format [, max_length]) date_string Campaign field that provides the date. input_format Date format of the Campaign field. output_format Date format in Engage. max_length (Optional) Limit the length of the date. The default is 32 characters. For example, you can define a derived field as, DATE_FORMAT(customer.purchDate, DELIM_M_D_Y, "%m/%d/%Y" ) . v customer.purchDate is the field in your customer database Chapter 5. Data upload - Campaign to Engage 47 v DELIM_M_D_Y describes a date format of month, day, year, separated by some type of delimiting character. The character can vary, depending on the database that supports your customer tables. v %m/%d/%Y defines the date output. This format displays March 17, 2014 as 03/17/2014. You can define derived fields in the Mail List process that generates the tab separated file for upload to Engage. Procedure 1. In the flowchart that selects the data that you want to upload to Engage, edit the Mail List process. 2. In the Mail List process, click the Personalization tab and click Derived Fields. The Create Derived Fields screen opens. 3. Give the derived field a name. 4. Click Formula Helper. Open the Date and Time Functions, and double click DATE_FORMAT. The DATE_FORMAT macro displays in the Expression field as DATE_FORMAT() 5. In the Available Fields section, select the field that provides the date that you want to upload. Click Use to add the field to the Expression field. 6. Configure the expression. DATE_FORMAT takes 3 required arguments. The arguments must appear inside the parentheses, as follows: DATE_FORMAT(<date_string>, <input_format>, "output_format") a. Move the field name to be inside the parentheses into the position defined as date_string. b. Define input_string. By default format is DELIM_M_D_Y. (Month, Day, Year – separated by a delimiter character). To see a list of possible date formats, see the embedded help in the Formula Helper for the DATE function. c. Define output_format. The output characters must appear between the quotation marks. For a list of output options, see the embedded help in the Formula Helper for the DATE_FORMAT function. The date format must be a format that is supported by Engage. d. Click Check Syntax to confirm that there are no errors in the expression. 7. Click OK. 8. On the Personalization tab, add the new derived field to the list of fields in the Export Fields field. Click OK. Results When you run the flowchart, the output file contains the date information in a format that Engage can accept. 48 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 6. Contact data download - Engage to Campaign Engage provides contact data that identifies records for individuals who have opted-in or opted-out of email messaging. Contact data also includes records for messages that might not be delivered successfully. The Campaign integration with Engage provides the contactDownload script that you can use to export contact data from Engage to Campaign. When you add the script to a flowchart as a flowchart trigger, you can schedule contact data downloads. Uploading data from Campaign to Engage involves the following general steps. v Decide what to download. The contactDownload script supports options that you can configure to specify the type of contact data that you want to download. The contact data is downloaded as a tab-separated file. v Decide where to save the download file. The download script provides an option to specify a directory. If you do not specify a directory, the script saves the file in the default download directory that is specified in the integration configuration properties. v Schedule the download. Configure a flowchart that includes a Schedule process that contains the contactDownload script as a trigger. Note: The Schedule process is used only to trigger the download of contact data as a file. Do not use this process to schedule flowchart runs. Use standard Campaign procedures to schedule a flowchart run. v Make contact data available in Campaign. You can include a Select process in the flowchart that you use to download the contact data. Configure the Select process to store the download file in a Campaign base table. Contact data available from Engage Engage makes contact data available that you can use to determine which message recipients have opted-in or opted-out. You can also download the entire contents of an Engage database or contact list that contains contact data. The contactDownload script includes options to specify the ID of the Engage database or contact list that provides contact data. You can also specify one of the following types of contact data to download. OPT_IN Contacts marked as opted in. OPT_OUT Contacts marked as opted out UNDELIVERABLE Contacts identified as undeliverable messages. Note: OPT_IN, OPT_OUT, and UNDELIVERABLE apply only to email messaging. SMS messaging uses a different method to manage SMS consent. © Copyright IBM Corp. 2014, 2017 49 Where to download contact data The contactDownload script downlands contact data from Engage as a tab-separated file and stores the file in a local directory. Specify the directory as an option in the contactDownload script or in the default location that you specify as a configuration property. The default location for downloaded contact data is the directory that you specify in the integration configuration for the property: local.download.dir. This property is contained in the configuration file that you can find at <CS_HOME>/conf/configuration.properties. You can also specify where to download contact data with the outputFile option in the contactDownload script. The directory structure must exist in advance of the download. If you do not provide a value for the outputFile option, the system saves the data in the default directory that you specify in the integration configuration properties. How to download contact data Downloading contact data from Engage to Campaign requires running a contact download script. You can run the script manually or automate the contact download process by incorporating the script into a Campaign flowchart. The result of running the download script is a tab-separated file that you save to a local directory. After the contact file is downloaded, you can select the contents of the file for use by Campaign. You can fully automate the process by scheduling a flowchart run that downloads the file and moves the file contents to a Campaign base table in the same flowchart run. contactDownload Run the contactDownload script to download contact data as a tab-separated file that can be used as input to a Campaign flowchart. To download contact data manually, run the contactDownload script from a command line on the Campaign Analytic server. To download contact data automatically according to a schedule, add the script to a flowchart process as a flowchart trigger in a Schedule process. The configuration of the contactDownload script determines what data to download, how much data to download, and where to save the tab-separated download output file. You must specify the Engage database, contact list, or query that provides the contact data. Specifying the type, quantity, and location of the downloaded data is optional. The contactDownload script is in the bin directory of the <CS_HOME> directory. Usage contactDownload -l <listID> [-t <exportType> -o <outputFile> -d <duration> -c <configPropertiesFile> -s <spPropertiesFile>] 50 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Option Description ID of the Engage database, contact list, or query that provided contact and personalization data for the mailing about which you are downloading contact data. Specify the ID as defined in the Engage interface. -l | --listID Defines the type of contact data to export from Engage. Specify one of the following (not case-sensitive): -t | --exportType ALL Export the entire database or contact list that you specify with the -l option. OPT_IN Contacts marked as opted-in. OPT_OUT Contacts marked as opted-out UNDELIVERABLE Contacts identified as undeliverable messages. Note: OPT_IN, OPT_OUT, and UNDELIVERABLE apply only to email messaging. -o | --outputFile Path, including file name, to the tab-separated file that contains the download. Directory structure must exist in advance of the download. If you do not provide a value for the -o option, the system saves the data in the directory that is specified in config.properties under local.download.dir. The system generates a file name as follows:Export_<listID>_<type>_<date:yyyymmddhhMMss>.TAB -d | --duration The system exports from Engage only records that were modified after a specific time. You specify the time by indicating the number of hours previous to the time of the current run of the contactDownload script. -c | --configPropertiesFile Path to the config.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location.[Default: <CS_HOME>/conf/configuration.properties.] Paths with spaces must be enclosed in quotation marks. -s | --spPropertiesFile Path to the sp.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/sp.properties.] Paths with spaces must be enclosed in quotation marks. Related concepts: “CS_HOME directory” on page 18 “SMS Opt-in and Opt-out synchronization between Campaign and Engage” on page 82 Related tasks: “Finding contact list, database, and relational table IDs in Engage” on page 31 “Finding mailing IDs in Engage” on page 32 “Downloading SMS consent data” on page 81 Downloading contact data by running a script To manually download contact data from Engage to campaign, run the contactDownload script from About this task Running the contactDownload script downloads a tab-separated file and stores it in a local directory that you specify either as a script parameter or as the default Chapter 6. Contact data download - Engage to Campaign 51 directory in the integration configuration properties. Procedure 1. Open a command line. 2. Change directory to the bin directory in your <CS_HOME> directory. 3. Run contactDownload.bat or contactDownload.sh with the options that you require. Help is available for the scripts on the command line. Enter -h or --help. What to do next After you run the script, you can use the downloaded data in various business processes, including IBM Campaign. You can automate the contact download process by incorporating the contactDownload script into a Campaign flowchart. Scheduling contact data downloads by running a flowchart Scheduled contact data downloads from Engage involve a series of procedures. Designing the flowchart defines what to download, configures a flowchart to run the download according to a schedule, and makes the downloaded data available in the Campaign database. About this task Automatically downloading contact data from Engage requires that you add a trigger to a Schedule process in a Campaign flowchart. The trigger contains the contactDownload script. How you configure the trigger determines what you download from Engage and where to save the downloaded data. Procedure 1. In Campaign, create or edit a flowchart. See the IBM Campaign User's Guide for information about creating and configuring flowcharts. 2. Add and edit a Schedule process. See “Configuring a Schedule process for contact data download” on page 54. 3. Define a trigger that contains the contactDownload script. See “Creating a flowchart trigger for contact data download” on page 53. 4. Add and edit a Select process. Configure the Select process to make the selected data available to downstream processes. See “Selecting downloaded contact data” on page 55. 5. Save and run the flowchart. You can schedule the flowchart to download contact data once or on a recurring basis. Note: You can also run the flowchart manually. For example, you might run the flowchart manually during the initial configuration to test the download performance. Results Running the flowchart triggers the following sequence. 52 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide 1. The Schedule process runs contactDownload as a trigger. You can configure the script to specify a name and location for the download output file, or you can accept the default name and location, as set in config.properties. 2. The contactDownload script retrieves the available contact data from Engage and creates the download output file in the default location or in the location that you specify with the -o option. 3. The Select process, which you map through a base table to the download output file, selects the downloaded data from the file, and passes it to downstream flowchart processes. What to do next Determine which downstream process to connect to the Select. For example, connect to a Segment process to create a segment so that other Campaign users and processes can use the data. For more information about flowcharts, see the IBM Campaign User's Guide. Flowchart design for downloading contact data You can create a Campaign flowchart to automate the process of downloading contact data and making it available in Campaign. You can create a flowchart that runs according to a schedule that you define, downloads contact data as a tab-separated file, and maps the file to a Campaign base table. The design of the flowchart includes the following elements. v Schedule process. The Schedule process lets you specify where to save the download file. See “Configuring a Schedule process for contact data download” on page 54. v Flowchart trigger. The flowchart trigger must be based on the contactDownload script. See “Creating a flowchart trigger for contact data download” v Select process. Use the Select process to map the download file to a Campaign base table. See “Selecting downloaded contact data” on page 55. Creating a flowchart trigger for contact data download In a Campaign flowchart, you can create a stored trigger that incorporates the contactDownload script. Before you begin You must have permissions in Campaign to create triggers. About this task When you configure automatic download of contact data from Engage, you specify a stored trigger that runs the contactDownload script. You can configure the contactDownload to download contact data from an Engage database, contact list, or query. The script creates a tab-separated file that you can map to a Campaign base record table for use by flowchart processes, such as a Select process. Chapter 6. Contact data download - Engage to Campaign 53 Procedure 1. Edit a Campaign flowchart. Open the Options icon Triggers. The Stored Trigger Definitions window opens. and select Stored 2. Click New Item. The data fields for the new trigger appear on the right of the window. 3. You can select a folder to save the trigger to in the Save Under list. The folder location governs which users can access the trigger, based on the folder's security policy. 4. Enter a name for the trigger in the Name field. v You cannot use spaces in the string, but you can use underscores ( _ ). v This name must be unique within the folder where you save it. 5. If you are creating a trigger in the top-level folder, select a security policy, or keep the default. 6. You can enter a description of the trigger in the Note field. You can provide a free-form text description of the trigger for documentation purposes. You also might want to keep a modification history of who modified the trigger, when, and what changes were made. 7. In the Command field, enter the contactDownload script. You must include the -l option to specify the database, contact list, or query that provides the contact data. You can specify a name and a location for the download file that the script generates. You can also specify options to control the type and amount of data that is downloaded. You might create several different triggers to perform different types of downloads. To illustrate the trigger configuration options, see the trigger examples. 8. Click Save and Close. Results When you configure a flowchart process that supports triggers, such as the Schedule process, the new trigger is available as a stored trigger. What to do next In a flowchart, configure a Schedule process with a trigger that is based on the contactDownload script. Configuring a Schedule process for contact data download In a Campaign flowchart, add a Schedule process that contains a flowchart trigger that runs the contactDownload script. The script downloads contact data from Engage and stores the downloaded data as a tab-delimited file that can be mapped to a Campaign table. About this task Place the Schedule process at the beginning of the flowchart so that it is the first process to run. You configure a Schedule process for automatic data download from Engage only so that it can trigger the contactDownload script after the process runs. You do not schedule anything when you use the Schedule process in this way. 54 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Procedure 1. Edit a Campaign flowchart. From the process palette, drag a Schedule process into the flowchart. 2. In the Schedule to Run field, select Once Only. 3. Select Send Trigger(s) After Each Run and select a trigger that is configured to download the type and quantity of data that you want, and that places the download file where you want it. You can select multiple triggers in a comma-separated list. The download triggers must be based on the contactDownload script. The configuration of the script determines the type, quantity, and location of the downloaded data. Note: Always place a question mark (?) after each trigger command so that, if the trigger fails, a red X displays on the Schedule process to inform you of the problem. 4. Click OK. What to do next In a flowchart, configure a Select process to select data from the tab-separated download output file. Selecting downloaded contact data The contactDownload script downloads contact data from Engage into a tab-delimited download output file. You can use a Select process to make the data in the download output file available in Campaign. Before you begin Locate the download output file that you want to use as input to the Select process. If you configured the contactDownload script with the -o option, go to the specified location. Otherwise, go to the default local download directory as specified in the local.download.dir property in config.properties, which is in the conf folder. About this task You must use the download output file as a data source for the Select process. Map the tab-delimited download output file to a base record table and then select the table as an input to the Select process. Procedure 1. Edit the flowchart where you added the Schedule process and contact download trigger. From the process palette, drag a Select process into the flowchart. 2. Map the download output file to a base record table. You must specify the following characteristics. v You are mapping to an existing file. v Select Delimited File. v The field delimiter is TAB. v Indicate that the first row of data contains field names. For more information about mapping a base record table to an existing delimited file, see the IBM Campaign Administrator's Guide. Chapter 6. Contact data download - Engage to Campaign 55 3. Select the base record table as an input to the Select process. 4. Connect the Select process to the Schedule process. What to do next Connect the Select process to downstream flowchart processes. Precautions for mapping the download output file to tables When you map a delimited file to a base record table, Campaign configures column widths automatically. Errors can result if the table widths are not appropriate for the data that is downloaded. Campaign detects field width automatically, based on the contents of the field. By default, Campaign examines the first and last 50 lines of a delimited file. It allocates field width based on the largest value it finds within those lines. However, in large delimited files, a subsequent field might exceed the estimated length, which can cause an error. To avoid possible column width problems, run the contactDownload script to populate the download output file. At least one row of the download file must contain data in every field so that Campaign can properly set column widths. For example, to download the list of individuals in an Engage database (37510 in this example), run the contactDownload script as follows. <CS_HOME>\bin\contactDownload.bat -l 37510 -t ALL After you populate the download output file, Campaign can allocate column widths more accurately, based on typical values. You can run the script later to download a more restricted set of records. Finding query IDs in Engage To use the contactDownload script to download query data from Engage, you must specify the query ID. You must access Engage to determine the appropriate query ID. Procedure 1. On the Data tab in Engage, go to Queries and click View. The View Data page opens and displays a list of queries as links. 2. In the list of queries, hold the cursor over a query name. The query ID displays in a tooltip. 56 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 7. Tracking data download - Engage to Campaign Engage records information that is related to the transmission, delivery, and response to the messages that you send through Engage. You can download the email tracking data that Engage collects by running the trackingDownload script. You can also use the UBX Toolkit to download tracking data for email, SMS, and mobile push messages. Run the trackingDownload script to download email tracking data from a single Engage database or all of the Engage databases that you create. The trackingDownload script downloads all tracking data that Engage received since the previous download. The trackingDownload script stores the downloaded tracking data in tables that you create when you install the integration files. The Campaign integration with Engage provides DDL scripts to create tables accept the tracking data that Engage provides. You can create these tables in the Campaign schema. To download tracking data for SMS messages and mobile push notifications, you can use the UBX Toolkit. The UBX Toolkit can download a tab-separated file that Engage creates to make SMS and push tracking data available. The UBX toolkit downloads the file to a local directory that you specify. You can also use the UBX Toolkit to download email tracking data, but the list of available results is smaller than the list that is available by running the trackingDownload script. You can run the trackingDownload script or the scripts in the UBX Toolkit manually according to a schedule. The integration tracks any errors that it encounters when it adds tracking data to local tables. To determine corrective action, you can review an error file and a generated file that lists rows that were not inserted into the database. You can also configure the system to automatically send reports by email to notify you when errors accumulate. Tracking data provided by Engage Engage tracks specific details about the messages that you compose and send through Engage. The Campaign integration with Engage makes that information available in the Campaign system tables so that you can use the data in Campaign flowcharts. During the initial setup and configuration of the integration, you run custom DDL scripts to add tables to the Campaign system tables. The trackingDownload script automatically downloads and inserts Engage tracking data into the tracking tables. By default, Engage tracks and can export the following types of message-related actions. v Messages sent v Messages opened v Messages that were forwarded v Clickthroughs © Copyright IBM Corp. 2014, 2017 57 v Attachments that were viewed or downloaded v Message bounces v Recipients for whom messages were suppressed v Record of Engage conversion events v Opt-in v Opt-out You can also configure the integration to track data that Engage does not track by default. However, to do so, you must modify the integration tracking tables that you created during the installation process and update the integration configuration. Adding additional tracking information You can configure the Campaign integration with Engage to download additional tracking information to Campaign. To receive the extra tracking data, you must modify the tracking tables that are created in the Campaign schema and update the integration configuration. About this task For example, to identify the audience level from Campaign, you might define an additional tracking field called customerID. To use customerID as an additional tracking field, you must add a customerID field to the integration tracking fields. You must also modify the integration configuration properties to specify the customerID as a field to be exported for additional tracking. Procedure v Ensure that the Engage database includes the tracking information that you want to add. v Modify the integration tracking tables in the Campaign schema. Use your database client to add the column that you want to add to all of the integration tracking tables (except sp.uploadaudit). The field name must be identical in the Campaign schema, Engage database, and in the integration configuration. v Modify config.properties to identify the additional field as a field that can serve a tracking field. 1. Go to <cs_home>/bin/config.properties. Open the file in a text editor. 2. Search for # export.additional.tracking.fields=. Remove the comment character # to enable the configuration. 3. Enter the column name of the additional field as the value for export.additional.tracking.fields. To add multiple fields, enter the column names as a comma-separated list. 4. Save your changes. Results Engage includes the additional field as one of the email-related elements that it tracks, and the additional data is downloaded and inserted into the integration tables. When you include the field in a data upload, Engage can return a value for the additional field as tracking data. If you do not include the additional tracking field in the upload, the field appears in the tracking data but the value in the extra column is null. 58 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Database management concerns and precautions Adding additional tracking fields to your database can have unintended effects. Consult with your database administrator to prevent problems. Be careful about how many fields that you add and how you configure each field. Consider the following issues. v Avoid adding fields where you expect to create one row for every message that you send. The wider that you make the table, the more space each row requires. v Do not add constraints to the fields that you add. v Do not add unique indexes on the fields that you add. v Reconsider how often you archive data or schedule data clean up. The added fields can increase the overall size of the database. The effect of the additional fields on your database depends on your business practices and messaging volumes. v Determine how often to run the trackingDownload script. Consider how often you send messages and the amount of tracking data that they might generate. Tracking data available through the UBX Toolkit The UBX Toolkit downloads tracking data as a series of events that are saved and downloaded as a tab-separated file. The types of events that the UBX Toolkit supports varies by communication channel. To support the use of the UBX Toolkit as a way to download message tracking data, Engage operates as an email and SMS event publisher in IBM Universal Business Exchange (UBX). To provide tracking data for push notifications, Xtify operates as a push notification event publisher in UBX. Campaign operates as an event subscriber in UBX for all channels. The UBX Toolkit provides a download script that you can run automatically or as a scheduled job to download tracking data in a tab-separated file for email, SMS, or mobile push notification messages. For more information on event data that is available through UBX, see the description of the UBX event catalog in IBM Knowledge Center at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBX/ Event_taxonomy_ubx/UBX_recognized_events.dita. Email tracking data available as an event Engage supports specific email events that provide tracking data for email messaging. The following table lists the email tracking data that Engage makes available as UBX events. The Event name might differ between mailings. The Event code must appear in the tracking data exactly as shown. Table 3. Email tracking events through UBX Event name Event code Mailing - sent emailSend Mailing - opened emailOpen Mailing - clicked emailClick Chapter 7. Tracking data download - Engage to Campaign 59 Table 3. Email tracking events through UBX (continued) Event name Event code Mailing - bounced emailBounce SMS tracking data available as an event Engage supports specific SMS events that provide tracking data for email messaging. The following table lists the SMS tracking data that Engage makes available as UBX events. The Event name might differ between programs. The Event code must appear in the tracking data exactly as shown. Table 4. SMS tracking events through UBX Event name Event code SMS - Sent from a SMS Program sentSMS SMS - Interacted with an SMS program interactedSMS Mobile push tracking data available as an event Engage supports specific mobile push events that provide tracking data for email messaging. The following table lists the mobile push tracking data that Engage makes available as UBX events. The Event name might differ between mailings. The Event code must appear in the tracking data exactly as shown. Table 5. Mobile push tracking events through UBX Event name Event code Mobile App - Installed application/appInstalled Mobile App - Uninstalled application/appUninstalled Mobile App - Opened a Push Notification simpleNotification/appOpened Mobile App - Clicked a URL simpleNotification/urlClicked Mobile App - Enabled a Push Notification application/uiPushEnabled Mobile App - Disabled a Push Notification application/uiPushDisabled Mobile App - Started a Session application/sessionStarted Mobile App - Ended a Session application/sessionEnded Where to download tracking data The trackingDownload script downloads contact data from Engage as a tab-separated file and stores the file in a local directory. The UBX Toolkit also saves tracking data to a local directory but by default stores it in a different location from where the trackingDownload script stores the download file. You specify the default destination for data downloaded when you install the integration files. The default location for downloaded tracking data is the directory that you specify in the integration configuration for the property: local.download.dir. This property is contained in the configuration file that you can find at <CS_HOME>/conf/configuration.properties. 60 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide You can specify a different tracking data download directory by changing the default location in configuration properties. Where the trackingDownload script stores downloads You specify the default destination for data downloaded when you install the integration files. The default location for downloaded tracking data is the directory that you specify in the integration configuration for the property: local.download.dir. This property is contained in the configuration file that you can find at <CS_HOME>/conf/configuration.properties. You can specify a different tracking data download directory by changing the default location in configuration properties. Where the UBX Toolkit stores downloads The default local download directory is $CU_HOME/AppData/downloads, but it can be customized in the config.properties file. $CU_HOME is the directory where you installed the UBX Toolkit. How to download tracking data The method that you use to download tracking data from Engage depends on the messaging channel and on the type of tracking data that you want to download. For email tracking data, you can run the trackingDownload script on the Campaign Analytics server or run the eventsDownload script in the UBX Toolkit. Running the trackingDownload script on the Campaign Analytics server returns a greater variety of tracking data. For SMS messages and mobile push notifications, run the eventsDownload script in the UBX Toolkit. You can run any of the scripts manually or as a scheduled job. trackingDownload Use trackingDownload to move tracking data from IBM Engage to local tables that you create during installation. Tracking data downloads include all tracking data received since the previous download. You can specify a single Engage database to view tracking data for only that database. If you do not specify a database, trackingDownload downloads all tracking data from all databases. By default, the trackingDownload script is located in the bin directory of the <CS_HOME> directory. The <CS_HOME> directory is the folder on the Campaign Analytics server where the integration files are installed. It can be any folder that you choose or create. You can run the trackingDownload script from a command line on the Campaign Analytics server. You can also configure a cron job or Windows scheduler task to run the script. You must have access to the Campaign Analytics server and user permissions that allow you to run scripts. Chapter 7. Tracking data download - Engage to Campaign 61 Usage trackingDownload [-l <listID> -c <configPropertiesFile> -s <spPropertiesFile> -j <jdbcPropertiesFile>] Option Description -l | --listID ID of the Engage database or contact list that provided contact and personalization data used in the mailing that you are tracking. Specify the ID as defined in the Engage interface. Note: Add the following in the config.properties file, when you are providing Engage database ID in parameter -l: export.additional.xml.elements=<INCLUDE_CHILDREN/><ALL_EVENT_TYPES/> export.additional.tracking.fields=CustomerID In place of CustomerID, you could specify columns from your database. To add multiple fields, enter the column names as a comma-separated list. The INCLUDE_CHILDREN directive will enable retrieving mailings for queries and contact lists based on the specified database. However, not all the customers will need to do this, but if they want to run the trackingDownload script with the "-l" parameter, they may have to do add the above settings in the config.properties file. -c | --configPropertiesFile Path to the config.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location.[Default: <CS_HOME>/conf/configuration.properties.] Paths with spaces must be enclosed in quotation marks. -s | --spPropertiesFile Path to the sp.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/sp.properties.] Paths with spaces must be enclosed in quotation marks. -j | --jdbcPropertiesFile Path to the jdbc.properties file. Absolute or relative path to <CS_HOME>. Specify a path when this file is renamed or is not in the default location. [Default: <CS_HOME>/conf/jdbc.properties. Paths with spaces must be enclosed in quotation marks. Related concepts: “CS_HOME directory” on page 18 Related tasks: “Encrypting the integration configurations” on page 24 “Finding contact list, database, and relational table IDs in Engage” on page 31 “Finding mailing IDs in Engage” on page 32 Downloading tracking data with a script You can run the trackingDownload script from a command line on the Campaign Analytics server. Run the script manually to verify that the configurations for the integration are properly set and to verify the connection to Engage. About this task When you run the trackingDownload script, the system indicates in the command console that the run was successful. Look in the console for any reported errors. 62 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Note: Running trackingDownload to verify system configurations and the connection to Engage is recommended before you encrypt the integration configurations. Procedure Run the trackingDownload script, as follows. <CS_HOME>\bin\trackingDownload.bat or <CS_HOME>/bin/trackingDownload.sh Results Script run results are reported explicitly when the run completes by printing a count of errors, warnings, and an overall status. Data that is received from Engage is added to the various integration tracking tables that you created locally during the installation process. Related concepts: “CS_HOME directory” on page 18 Related tasks: “Encrypting the integration configurations” on page 24 Scheduling tracking data downloads You can automatically download tracking data from Engage by running the trackingDownload script as a cron job or Windows scheduler task. Before you begin Before you run the trackingDownload script, ensure that the required tracking tables are available in your local environment. To create the tables, run the custom DDL scripts that are provided with the integration download package. About this task Each time the trackingDownload script runs, it downloads various types of tracking data and inserts into the appropriate tracking tables. Each download includes all of the tracking data that was received since the previous run of the script. Depending on how you configured the script, the download includes data from all available databases, or a single database that you specify in the script. Procedure Run the trackingDownload script as a cron job or Windows scheduler task. In Windows, ensure that your action uses cmd /c: and use double quotation marks around the entire command. For example: "cmd /c C:\IBM\EMM\Campaign\partitions\partition1\campSP\bin\ trackingDownload.bat" Tracking data download with the UBX Toolkit To download tracking data for SMS messages and push notifications, you must use the UBX Toolkit. Chapter 7. Tracking data download - Engage to Campaign 63 The UBX Toolkit provides a way for locally installed applications, like IBM Campaign, to interact with IBM Universal Behavior Exchange (UBX). UBX creates a structure that supports dynamic relationships between independent software applications, like IBM Engage, that register with UBX. Each UBX participating application can provide different types of marketing data and different ways to identify individual customers. The UBX Toolkit installs behind your corporate firewall to securely connect your local applications and databases to UBX APIs and the IBM Commerce ecosystem. When Engage receives tracking data as a result of sending email, SMS, or push notification messages, it publishes the data as email, SMS, or mobile events to UBX. UBX temporarily stores the event data as a tab-separated file. The UBX Toolkit provides the eventsDownload script to download the tab-separated file to a directory in your local environment. You can run the script manually or as a scheduled job. Downloading tracking data with the UBX Toolkit involves the following general steps. v Register Campaign and Engage as UBX endpoints. To integrate Campaign and Engage, register Engage as an event publisher and Campaign as an event consumer. See “Preparing UBX endpoints for the integration” on page 27. v Subscribe to the events that Engage publishes for each messaging channel that you use. For information about the event data that Engage provides, see “Tracking data available through the UBX Toolkit” on page 59. v In the UBX Toolkit, run the eventsDownload script manually, or as a scheduled job. The script downloads the available tracking data as a tab-separated file (.tsv). For more information about how to run this script, see the instructions on IBM Knowledge Center for downloading events from UBX at http://www.ibm.com/support/knowledgecenter/SS9JVY/UBXtoolkit/ Operation_toolkit/Downloading_events_from_UBX.dita UBX Overview IBM UBX is a cloud-based service from IBM that you can use to selectively exchange business data between various IBM Commerce solutions and IBM Business Partner applications. Owners of IBM Commerce and IBM Business Partner solutions can use UBX to specify what to share and when to share it. By blending data contributions from multiple UBX endpoints, UBX users can build a view of their customers that is richer, better targeted, and more timely than any single application can provide. Participating IBM Commerce and IBM Business Partner solutions can quickly integrate data from various IBM Commerce applications that they install in their own computer network or that they access as a web-based service. They can also integrate data from selected independent companies who collaborate with IBM to make their customer interactions available through UBX. Marketing applications from IBM and selected IBM Business Partners access UBX automatically through various APIs. Each application is referred to as an endpoint. 64 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Each endpoint must be registered with UBX. When you register an endpoint with UBX, the endpoint appears as an option in the UBX user interface. UBX users create connections between data that is produced by registered applications to help address their unique use cases. These connections are referred to as syndications. In UBX, syndication refers to the process that business users complete to select a data provider, specify the data to exchange, and designate where to deliver the data. The following diagram illustrates the relationships and interactions between UBX and the syndicated marketing applications that participate in UBX. For more information about how to integrate business solutions through UBX, see the IBM Universal Behavior Exchange Integration Guide. Error reporting for tracking data In some cases, tracking data cannot be inserted into the tracking tables in Campaign. The trackingDownload script stores the rows that cannot be downloaded in a file. Rows that cannot be written to the tracking tables are stored in <CS_HOME>/AppData/trackingDataError. Depending on how many errors you encounter, the file can become large. The system sends an error report by email when the amount of error data exceeds a configured threshold. You can also save files that were downloaded and successfully inserted into the database. However, saving all of the successful downloads might require careful management of your disk space, due to the potentially large number of records. Chapter 7. Tracking data download - Engage to Campaign 65 The Campaign integration with Engage provides several configurations to manage tracking data error handling and reporting. The configurations are in the config.properties file, which is in the bin directory of the integration files. You can use the following configuration settings to manage error reporting for your installation. Note: If the SMTP server that receives and transmits error reports requires authenticated access, you can add authentication credentials in system configuration properties to enable the integration to access the server automatically. Error files are saved in a compressed format. Configuration property Description dbinsert.errors.report.alert.records.num Threshold for sending error alerts. dbinsert.errors.report.alert.error.files.size Specify the cumulative size of the accumulated error alerts that triggers a report. dbinsert.errors.report.smtp.host Specify the server that sends SMTP alerts. dbinsert.errors.report.smtp.auth.user The user name for SMTP authentication, if required by the SMTP server that you use to send alerts. Enables the integration to submit errors automatically. dbinsert.errors.report.smtp.auth.password The password for SMTP authentication, if required by the SMTP server that you use to send alerts. Enables the integration to submit errors automatically. dbinsert.errors.report.to Specify email address to receive alerts. dbinsert.errors.report.from Specify a From: address for alert messages. dbinsert.errors.report.body Content of alert messages. dbinsert.errors.report.subject Subject line for alert messages dbinsert.keep.processed.files Indicate if the system stores all processed files locally dbinsert.errors.report.class Specifies a class to be called when the download script completes. local.download.dir Local directory that receives downloaded tracking data. 66 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 8. Email messaging Through the integration of Campaign and Engage, you can segment and upload contact information from Campaign that you can use to create and send mailings from Engage. The integration provides the following email messaging options. v Data upload only. Upload contact data from Campaign to an Engage database, contact list, or relational table. Create and send mailings in Engage separately. See “Uploading email contact data to Engage.” v Data upload that triggers a pre-configured mailing in Engage. Specify a mailing template that you have created in Engage. When you upload data from Campaign, the integration script triggers the mailing, based on the mailing configuration and scheduling that you define in Engage. See “Sending email messages after uploading contact data” on page 70. v Download email recipient contact data. Run a Campaign flowchart that contains an integration script that downloads contact data from Engage and adds it to a Campaign table. “Downloading email contact data” on page 72. v Download tracking data. Run a Campaign flowchart that contains an integration script that downloads data that relates to the transmission, delivery, and response to the messages that you send through Engage. See “Downloading email tracking data” on page 73. Uploading email contact data to Engage Create and run a Campaign flowchart that selects data that describes and provides contact information for a target audience. Configure the flowchart to automatically upload the data to an Engage database or contact list when the flowchart run completes. To use a Campaign flowchart to upload contact data to an Engage database or contact list, add the contactUpload script to a trigger in a Mail List process at the end of the flowchart. To use a Campaign flowchart to upload contact data to an Engage relational table, add the tableUpload script to a trigger in a Mail List process at the end of the flowchart. Uploading email contacts to an Engage database To configure a flowchart to automatically upload Campaign data to an Engage database, add a flowchart trigger that contains the contactUpload script to the Mail List process. The script must reference the database ID. v Trigger design contactUpload -i <inputFile> -l <databaseID> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 © Copyright IBM Corp. 2014, 2017 67 When the flowchart run finishes, the trigger runs the script and the system uploads the tab separated input file to the specified Engage database. Uploading email contact data to a contact list To upload Campaign data to a single Engage contact list, run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger. You must specify the ID or name of the contact list and the parent database of the contact list. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListID> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 When the flowchart run finishes, the trigger runs the script and the system uploads the tab separated file that contains the flowchart output to the Engage database. Only the database rows that are specified in the contact list are modified. Uploading email contact data to a contact list identified by name You can run the contactUpload script to upload data to an existing Engage contact list that you identify by name, instead of by list ID. Run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contact list name> v Example <CS_HOME>\bin\contactUpload.bat -i\<a folder>\FC_out.tsv -l 12345 -p CustomerAprilBday In this example, the flowchart logic selects customers that celebrate birthdays during the month of April so that you can send a Happy Birthday email. The contact list that is specified by (–p) is not unique, but it is easy to remember that it is one of twelve lists that you might create in advance in an Engage database. When the flowchart runs, the contactUpload script finds an existing contact list with the same name. The script uploads the customer data to the existing contact list. Uploading email contacts and creating a new contact list When you upload data from Campaign to Engage, you can create a new contact list by specifying a contact list name that is not already used within your Engage organization. Run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contact list name> v Example <CS_HOME>\bin\contactUpload.bat -i\<a folder>\FC_out.tsv -l 12345 -p "newCust_wkOf 4.1.15" 68 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide In this example, the flowchart logic selects new customers that were registered during the previous week so that you can send a welcome email. Because the contact list name that is specified by (–p) includes a specific date, the name does not match any existing contact list. When the flowchart runs, the contactUpload script cannot find an existing contact list with the same name. It calls an Engage API to create a new contact list to receive the data for the new customers. If you run the same flowchart again, the new upload overwrites the existing list because the script now sees a name match and uploads the data to the list. Notice that because the new contact list name includes a space, you must enclose the name in quotation marks. Uploading email contacts to multiple contact lists To upload Campaign data to multiple Engage contact lists, run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger. Specify the multiple contact lists by entering a comma-separated list of contact list IDs as a parameter for contactUpload. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765,43211,10298 Do not include spaces after the commas when you specify contact list IDs for the – p option. When you run the flowchart, the system uploads the tab-separated file that contains the flowchart output to the Engage database. Only the database rows that are specified in the various contact lists are modified. Uploading email contact data as specified in a mapping file When you upload Campaign data to Engage, you can use a mapping file to map field names in Campaign to fields in an Engage database or relational table. In the file, you specify the target database, contact list, or relational table. Optionally, you can specify multiple contact lists for the upload. v Trigger design contactUpload -i <inputFile> -m <mapping_file> v Example For upload to a contact list: <CS_HOME>\bin\contactUpload.bat -i ..\<sub-folder>\FC_out.tsv -m ..\<sub-folder>\map_FC_out.xml For upload to a relational table: <CS_HOME>\bin\tableUpload.bat\ -i\<sub-folder>\FC_out.tsv -m ..\<sub-folder>\map_FC_out.xml When the flowchart run finishes, the system uploads the tab separated file that contains the flowchart output to Engage. If the mapping file specifies contact lists Chapter 8. Email messaging 69 as the upload destination, only the rows that are specified in the various contact lists are modified. Sending email messages after uploading contact data Create and run a Campaign flowchart that uploads contact data to Engage and then triggers a mailing in Engage. You must build and configure the mailing in Engage before you run the flowchart. To trigger the mailing in Engage when the upload completes, add the contactUpload script to a trigger in a Mail List process at the end of the flowchart and specify a mailing template ID as a script parameter. Sending a mailing after you upload data to a contact list You can upload Campaign data to an Engage contact list and automatically send an Engage mailing that references the updated contact list. To trigger the mailing in Engage when the upload completes, specify a mailing template ID when you configure the contactUpload script. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> -e <templateID> To determine the mailing template ID, log in to Engage, go to the list of mailings, and hold the cursor over the mailing name. The ID displays in a tooltip. v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 -e 45678 When the flowchart run finishes, the system uploads the tab separated file that contains the flowchart output to Engage. When the upload is complete, Engage processes the new contact list and transmits messages to the specified recipients, based on the information in the updated contact list. Upload to a contact list and trigger a mailing with a modified mailing name You can configure the contactUpload script to trigger a mailing that is identified by additional information in the mailing name. You can run the contactUpload script that triggers a mailing from several different flowcharts or from the same flowchart at multiple times. Each flowchart run might generate a different segment or can be associated with a different campaign. To distinguish the resulting mailings, you can append additional information to the mailing name. The script also appends a timestamp to the name. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> -e <templateID> -k <campaigncode> v Example To send and identify a limited time promotional mailing to Gold customers, configure a flowchart trigger as follows. 70 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 -e 45678 -k goldCust When the flowchart runs, the script uploads selected customer data and triggers a preconfigured personalized email to Gold customers. The mailing name format is <campaign code> <mailing template name> <timestamp>. For example, goldCust WeeklyDealsNSteals 04:01:15:01:59. When Gold customers respond, you can download the contact data. The unique mailing name is available in the MailingName field in the integration tracking tables to support additional post-processing. Upload to multiple contact lists and trigger a mailing You can upload Campaign data to multiple Engage contact lists and automatically send an Engage mailing that references one of the updated lists. To trigger the mailing in Engage when the upload completes, specify a mailing template ID when you configure the contactUpload script. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> -e <templateID> -g <recipientListID> v Example ..\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765,43211,10298 -e 45678 -g 43211 Do not include spaces after the commas when you specify contact list IDs for the – p option. After the flowchart run finishes, the system uploads the tab-separated file that contains the flowchart output to the Engage database. Only the rows that are specified in the various contact lists are modified. When the upload is complete, Engage updates all of the contact lists that are specified with the -poption and sends the mailing that you specify with the -e option to the contact list that you specify with the -g option. Upload mapped data and trigger a mailing When you upload Campaign data to Engage and specify a mapping file, you can also trigger an Engage mailing after the upload completes. This option is available only when you upload to a single contact list. v Trigger design contactUpload -i <inputFile> -m <mapping_file> -e <templateID> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<subfolder>\FC_out.tsv -m ..\<subfolder>\map_FC_out.xml -e 45678 When the flowchart run finishes, the system uploads the tab separated file that contains the flowchart output to Engage. If the mapping file specifies contact lists as the upload destination, only the rows that are specified in the various contact lists are modified. Note: If the mapping file references multiple contact lists, you must use the -g option to indicate which contact to use with the mailing that you specify with the -e option. Chapter 8. Email messaging 71 When the upload is complete, Engage processes the new contact list and sends the mailing that you specified with the -e option, based on the information in the updated contact list. Downloading email contact data The integration of Campaign and Engage includes the contactDownload script. You can run the script manually or as part of a flowchart trigger to download contact data as a tab-separated file that can be used as input to a Campaign flowchart. Preparation and procedures for downloading contact data depends on the configuration of the trigger that you add to a Schedule process in a flowchart. The trigger is based on the contactDownload script. How you configure the script depends on the type of data that you want to download and where you want to store the downloaded data. The contactDownload script provides several options for downloading email contact data. Downloading all available contact data You can run the contactDownload script to download all of the available contact data in an Engage database, contact list, or query. The ID of the database, list, or query is a required input to contactDownload. If you run contactDownload and do not specify an export type, the script downloads all available data for the specified contact list. However, you can also explicitly download all of the contact data in a database or contact list by specifying ALL as the export type. v Trigger design contactDownload -l <listID> contactDownload – l <listID> -t ALL (optional) v Example <CS_HOME>\bin\contactDownload.bat -l 12345 Defining a static output file name and custom location When you run the contactDownload script, you can use the -o option to specify where to save the tab-separated download file, instead of the default download directory. By default, the local download directory is <CS_HOME>/AppData. In the integration configuration, the local.download.dir property defines a location for the file, if you do not specify a location with the -o option. The system generates a name for the contact download file in the following format. The time stamp changes every time you download. Export_<listID>_<export_type>_<yyyymmddhhMMss>.TAB For example: Export_12345_OPT_OUT_20140704091533.TAB If you configure a flowchart to download contact data automatically, you can use the -o option to define a static file name that you can specify in the Schedule process trigger. 72 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide v Trigger design contactDownload – l <listID> -o <outputFile> v Example <CS_HOME>\bin\contactDownload.bat -l 12345 -t OPT_IN -o ..\email_ADD\Export_OptIN.tab In this example, records for individuals who want to be contacted by email are included in the Export_OptIN file, which is saved in the email_ADD folder. This folder is an example of a custom folder. It is not part of the default directory structure. Downloading a specific type of contact data When you run the contactDownload script, you can specify the type of data to download. Only records that are marked as the type of data that you specify are added to the tab-separated download file. You specify the type of contact data to download with the -t option. This option takes the following values. v ALL v OPT_IN v OPT_OUT v UNDELIVERABLE Note: OPT_IN, OPT_OUT, and UNDELIVERABLE apply only to email messaging. v Trigger design contactDownload – l <listID> -t <export_type> v Example <CS_HOME>\bin\contactDownload.bat -l 12345 -t OPT_OUT Downloading contact data received during a specific time When you run the contactDownload script, you can specify how much data to download by limiting results for a specific number of hours previous to running the script. Only records that are modified during the time that you specify are added to the tab-separated download file. You limit the time to include in the download with the -d option. You can use this option in conjunction with other options to narrow your results. v Trigger design contactDownload – l <listID> -d <duration> v Example <CS_HOME>\bin\contactDownload.bat -l 12345 -t OPT_OUT -d 168 The script generates the tab-separated download file in the default download directory. In this example, the file contains only opt-out records that changed during the previous week (168 hours = 1 week). Downloading email tracking data The integration of Campaign and Engage includes the trackingDownload script. The trackingDownload script stores the downloaded tracking data in local tables. Chapter 8. Email messaging 73 You can run the trackingDownload script from a command line on the Campaign Analytics server. You can also configure a cron job or Windows scheduler task to run the script. Depending on how you have configured the script, the download includes data from all available databases, or a single database that you specify in the script. Download tracking data from a specific Engage database To download tracking data from a specific Engage database, specify the database ID with the -l option. Specify the database that provided contact and personalization data that is used in the mailing that you are tracking. v Script design trackingDownload -l <listID> v Example <CS_HOME>\bin\trackingDownload -l 12345 In this example, the script downloads all available tracking data from the Engage database that is identified with ID 12345. Downloading tracking data from all Engage databases To download tracking data from all available Engage databases, do not specify a specific database ID with the -l option. Run the trackingDownload script, as follows. <CS_HOME>\bin\trackingDownload Support for Send Time Optimization When you upload contact data to Engage, you can include a parameter that instructs Engage to enable Send Time Optimization for the Engage database. When you enable Send Time Optimization (STO) for a database, Engage sends each message at a specific time of day and day of the week for each recipient in the database, based on the previous behavior of each individual. Instead of sending all email messages when you upload the contact list, Engage sends each message at a time when the intended recipient is most likely to open it. To enable STO, include the -n | --sto option in the contactUpload script when you upload the contact list to Engage. For more information about Send Time Optimization, see the IBM Marketing Cloud Knowledge Base and review the description for Send Time Optimization. Filter mailing templates If you have a large repository of mailing templates in IBM Engage, you can increase the efficiency of contact upload by setting a filtering framework. The filtering framework uses either date and time, or mailing template name and ID, to batch and sift through mailing templates. To set up a filtering framework, you must configure several properties in the config.properties file. 74 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Table 6. Filter by time and date Property Value Description mailingTemplates. filterMailingTemplatesBylmd True | False If set to true, the framework begins filtering mailing templates from the last modified date. In some cases, this can be the current date. mailingTemplates. initDate MM/dd/yyyy This property is required if mailingTemplates. filterMailingTemplatesBylmd is set to true. This property is used to communicate the initialization date for your IBM Engage account. When configured, the framework returns mailing template results up to this date. mailingTemplates. interval Nonzero, positive number This property is required if mailingTemplates. filterMailingTemplatesBylmd is set to true. This property sets the interval, in months, that the filtering framework uses to limit results. Based on the interval, the framework counts backward from the current month. For example, if the interval is set to 3 and the current date is 06/01/2015, the framework looks through all mailing templates from 03/01/2015 to 06/01/2015. If it cannot find the template in that date range, it then looks back another 3 three months. It continues searching until it finds the template, or comes to the initialization date. Table 7. Filter by mailing template name and ID Property Value Description schedulemailing. mailingname %s[mailingName] Configure this property to search for a mailing template by name. You must input "%s" before the mailing name for this property to be configured properly. For example, schedulemailing. mailingname=%s Your mailing name. If this property overrides mailingTemplates. filterMailingTemplatesBylmd. Chapter 8. Email messaging 75 76 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 9. SMS messaging Through the integration of IBM Campaign and IBM Engage, you can upload audience data, including SMS address and consent data, from Campaign to Engage. Use the SMS Campaign Manager to send SMS messages that rely on the uploaded data. The integration provides resources for the following typical SMS messaging scenarios. v Upload SMS consent and address data from your marketing databases to Engage for use in SMS programs to send SMS messages. See “Uploading SMS contact and consent data to Engage” on page 79. v Send SMS messages automatically after you upload SMS information that you select from your marketing databases. See “Sending SMS messages after uploading SMS contact and consent data” on page 80. v Download SMS contact data from Engage to Campaign See Downloading SMS message contact data v Download tracking data from Engage to Campaign See “Downloading SMS message tracking data” on page 83 General requirements for SMS messaging To contact your customers with SMS messages through Engage, you must satisfy certain requirements and become aware of restrictions that are related to SMS messaging. For more information about SMS messaging through Engage go to https://kb.silverpop.com/kb/Engage/SMS_2.0/. Purchase requirement for SMS messaging To send SMS messages through Engage, you must purchase SMS support separately. When you purchase support for SMS messaging, IBM Marketing Cloud provisions your account for SMS messaging. Contact your IBM representative to learn more about how to purchase SMS support and begin the provisioning process. SMS number format When you specify a phone number for SMS messaging, you must provide the number in the correct format. Because SMS messages can be directed anywhere in the world, you must provide the phone number in international format. Review the SMS contact information in your customer database to ensure that SMS phone number information for each recipient. The phone number, including numbers for recipients in the United States, must include the international country code. © Copyright IBM Corp. 2014, 2017 77 To send SMS messages through Engage, the SMS number cannot include non-numeric characters. The elements of the number cannot be delimited by dashes, periods, or slashes. SMS database requirement In Engage, your organization can create only one database for SMS messaging. When you run the contactUpload and contactDownload scripts for SMS data, you must specify the dedicated SMS database. The database for SMS must be a non-keyed Engage database. SMS program requirement: Text to Join To send SMS messages through Engage, you must configure an SMS program in Engage. For example, use a Text to Join program in Engage. Consent requirements for SMS messaging SMS is an international communication medium. It is subject to various international laws and conventions. You must familiarize yourself with the applicable laws that pertain to contact by SMS in the markets that you want to reach. For example, you cannot contact individuals in the United States without obtaining permission in advance for contact by SMS messages. It is a violation of MMA Compliance to send SMS messages without proper consent from the recipient. Although you can upload SMS contact information from your customer tables through Campaign, you cannot send SMS messages unless you also provide SMS consent data. For more information about how to manage SMS consent, see the description in the IBM Marketing Cloud Knowledge Base for how to use and manage SMS consent at https://kb.silverpop.com/kb/Engage/mobile/SMS__Silverpop_Mobile_Messaging/001_How_To/9_Use_and_Manage_SMS_Consent. Data required for SMS consent SMS messaging requires recipient consent. A record in an Engage SMS database has one of the following statuses. v no consent v opted-in v opted-out If you upload contact data without also uploading the required consent, the record is marked no consent. The contact data that you provide for sending SMS messages must include a consent field that provides the CONSENT_STATUS_CODE. The only allowed values for CONSENT_CODE are OPTED-IN or OPTED-OUT. The values must be in uppercase and written in the exact phrase. Engage can send SMS messages only to recipients whose record is marked OPTED-IN. 78 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Optionally, if your database records the date and method of consent, you can define fields for the following fields. v CONSENT_DATE: Date on which consent was updated. The default value is the date of the upload. v CONSENT_SOURCE: Text description of where the consent was obtained. The default value is List Import. Related concepts: “What to upload from Campaign” on page 30 Related reference: Chapter 11, “Reference: Mapping files,” on page 89 Uploading SMS contact and consent data to Engage Create and run a Campaign flowchart that selects data that provides contact information for a target audience, including SMS consent status. To upload SMS data to Engage, run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger in a Mail List process at the end of a flowchart. The uploaded file must contain the CONSENT_STATUS field. Uploading SMS data to an Engage database The script must reference the database ID. v Trigger design contactUpload -i <inputFile> -l <databaseID> -u <channel> -b <sms program ID> -f <sync field> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -u sms -b 116962 -f "Mobile Number" When the flowchart run finishes, the trigger runs the script and the system uploads the tab separated input file (.tsv) to the specified Engage database. Uploading SMS data to a contact list The script must reference the contact list ID and the parent database ID. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListID> -u <channel> -b <sms program ID> -f <sync field> v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 -u sms -b 116962 -f "Mobile Number" When the flowchart run finishes, the trigger runs the script and the system uploads the tab-separated file that contains the flowchart output to the Engage database. The uploaded file must contain the CONSENT_STATUS field. Only the database rows that are specified in the contact list are modified. Chapter 9. SMS messaging 79 Uploading SMS data to a contact list identified by name Run the contactUpload script to upload SMS data to an existing Engage contact list that you identify by name, instead of by list ID. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contact list name> -u <channel> -b <sms program ID> -f <sync field> v Example <CS_HOME>\bin\contactUpload.bat -i\<a folder>\FC_out.tsv -l 12345 -p CustomerAprilBday -u sms -b 116962 -f "Mobile Number" In this example, the flowchart logic selects customers that celebrate birthdays during the month of April so that you can send a Happy Birthday text message by SMS. When the flowchart runs, the contactUpload script finds an existing contact list with the same name. The script uploads the customer data to the existing contact list. Uploading SMS data and creating a new contact list When you upload data from Campaign to Engage, you can create a new contact list by specifying a contact list name that is not already used within your Engage organization. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contact list name> -u <channel> -b <sms program ID> -f <sync field> v Example <CS_HOME>\bin\contactUpload.bat -i\<a folder>\FC_out.tsv -l 12345 -p "newCust_wkOf 4.1.15" -u sms -b 116962 -f "Mobile Number" In this example, the flowchart logic selects new customers that were registered during the previous week so that you can send a welcome message over SMS. Because the contact list name that is specified by (–p) includes a specific date, the name does not match any existing contact list. When the flowchart runs, the contactUpload script cannot find an existing contact list with the same name. It calls an Engage API to create a new contact list to receive the data for the new customers. If you run the same flowchart again, the new upload overwrites the existing list because the script now sees a name match and uploads the data to the list. Notice that in this example the new contact list name includes a space, so you must enclose the name in quotation marks. Sending SMS messages after uploading SMS contact and consent data Create and run a Campaign flowchart that uploads SMS contact and consent data to Engage and send SMS messages to the list according to a program that is defined in Engage. You must build and configure the SMS program in Engage before you run the flowchart. 80 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide You must build and configure the SMS program in Engage before you run the flowchart. To upload SMS data to Engage, run the contactUpload script from a command line on the Campaign Analytics server or as part of a flowchart trigger in a Mail List process at the end of a flowchart. The uploaded file must contain the CONSENT_STATUS field. Send SMS messages after uploading SMS contact and consent data to a contact list You can upload SMS data from Campaign to an Engage contact list and automatically send SMS messages based on the updated contact list. Specify a mailing template ID and the SMS program ID when you configure the contactUpload script. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> -e <templateID> -u <channel> -b <SMS program ID> -f <sync field> To determine the mailing template ID, log in to Engage, go to the list of mailings, and hold the cursor over the mailing name. The ID displays in a tooltip. v Example <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 -e 6e912c9d-aa88-49f1-82c6-9d3f7f96ac9c -u sms -b 116962 -f "Mobile Number" When the flowchart run finishes, the system uploads the tab separated file that contains the flowchart output to Engage. If the mapping file specifies contact lists as the upload destination, only the rows that are specified in the various contact lists are modified. When the upload is complete, Engage processes the new contact list and transmits SMS messages to the specified recipients, based on the information in the updated contact list. Downloading SMS consent data To download SMS consent data from Campaign to Engage, use the contactDownload script. Select consent data for download with queries that you design and run in Engage. Before you begin In Engage, create and run a query to extract the consent data that you want to download. For example, use a query to identify recipients who have opted-out of SMS messaging, or a list of SMS recipients that responded to a program. For more information about how to structure and run a query for SMS data, see https://kb.silverpop.com/kb/Engage/SMS_2.0/. Each query has a unique ID. You can reference the query ID in the contactDownload script to download the query results. You must access Engage to determine the query ID. In the list of queries, hold the cursor over the query name. The query ID displays as a tooltip. Chapter 9. SMS messaging 81 About this task To download consent data from Engage to Campaign, performs series of tasks. 1. Create and run a query to select the consent data that you want to download. 2. Create and run a Campaign flowchart to download the data from Engage. 3. Move the downloaded data to Campaign tables where it can be selected for use by various Campaign processes. For more information about creating and running flowcharts, see the IBM Campaign User's Guide. Procedure 1. In Campaign, create a flowchart or edit an existing flowchart. Define a trigger that contains the contactDownload script. For more information, see “Creating a flowchart trigger for contact data download” on page 53. 2. Add a Schedule process to the flowchart. Add the trigger from Step 1 to the process. For more information, see “Configuring a Schedule process for contact data download” on page 54. 3. Map the downloaded consent data to a base table in Campaign. Locate the downloaded file in the default local download directory as specified in the local.download.dir property in config.properties (in the conf folder) or in the directory that you specify with the -o option. 4. Add a Select process to the flowchart. Select the base record table as an input to the Select process and connect the Select process to the Schedule process. Connect the Select process to downstream flowchart processes. For example, connect to a Segment process to create a segment so that other Campaign users and processes can use the data. 5. Save and run the flowchart. What to do next To download contact data on a recurring basis, you can run the flowchart according to a schedule. You can also run the flowchart manually. For example, you might run the flowchart manually during the initial configuration to test the download performance. Related reference: “contactDownload” on page 50 SMS Opt-in and Opt-out synchronization between Campaign and Engage To ensure that consent records for SMS are as up to date as possible, you can update opt-in and opt-out requests for SMS that you receive through various channels. To synchronize SMS subscription data between Campaign and Engage, upload and download opt-in and opt-out updates regularly. Managing opt-in and opt-out records for SMS requires specific steps. The OPT_IN and OPT_OUT options for the contactUpload and contactDownload scripts do not apply to SMS messaging. 82 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide The first time that you add contact information for a recipient, the record is marked as an Opt-in record. If the individual did not consent to be contacted by SMS, you must subsequently mark the record as an Opt-out. You cannot add a record as an Opt-out record. You can identify a record as an Opt-out only after you enter it as an Opt-in. To keep SMS subscriptions up to date, you can schedule Campaign flowcharts that trigger the contactUpload and contactDownload scripts to run automatically. In Engage, schedule queries that update opt-in and opt-out status so that the most current information is available for download to Campaign. Related reference: Chapter 11, “Reference: Mapping files,” on page 89 “contactUpload” on page 33 “contactDownload” on page 50 Downloading SMS message tracking data You must use the UBX Toolkit to download SMS message tracking data from Engage to Campaign. Before you begin Register Campaign and Engage as UBX endpoints. Register Engage as an event publisher and Campaign as an event consumer. In UBX, subscribe Campaign to the SMS events that Engage publishes. About this task To download SMS tracking data, you must run the eventsDownload script to download event data from UBX to a local directory. You can run the script manually or as a scheduled job. By default, the script places the file in ../AppData/downloads where you installed the UBX Toolkit, but you can specify a different default directory in the config.properties file. The script downloads the available tracking data as JSON and converts it to a tab-separated file (.tsv). Both files are saved. After you download the tracking data to a local directory, run the eventsImport script to add event data from the TSV file to a local SQL database. You can also run this script manually or as a scheduled job. You can insert a single file or a single directory into the database. The UBX Toolkit provides a mapping file to specify how the event data is added to the database. Confirm that the SMS tables and the UBX Toolkit mapping file display the same event names as the event that you download. Examine the database after you download several events and adjust the database tables and the UBX Toolkit mapping file. Procedure 1. Run the eventsDownload script manually, or as a scheduled job. Chapter 9. SMS messaging 83 eventsDownload [-c <config properties file>] Use the -c option if you use a configuration properties file that is not in the default location (conf directory) or that does not use the default file name (config.properties) Example: eventsDownload.bat [-c <config properties file>] For more information about how to run the script, see the instructions on IBM Knowledge Center for downloading events from UBX at: Downloading events from UBX. The UBX toolkit downloads event data from UBX in JSON format. When the event data finishes the download, the eventsDownload script converts JSON objects to TSV format. 2. Run the eventsImport script. Specify the mapping file that you created for the UBX Toolkit. eventsImport -m <mapping file> -i <source TSV file> -f <source directory> The default mapping file is EventsToDBMapping.xml. Example: eventsImport.bat -m EventsToDBMapping.xml -i SMSsent -f ToolKit_home/AppData/downloads For more information about how to run the script, see see the instructions in IBM Knowledge Center at: Importing event data into a database Results After the eventsImport script imports the event data to the specified database, the script moves the TSV files to the <UBX Toolkit home directory>/AppData/ dataProcessed directory. The dataProcessed folder is created under the downloads folder that you specify in config.properties. What to do next To download tracking data on a recurring basis, you can run the script according to a schedule. Use Campaign flowcharts to select the tracking data for use in Campaign for retargeting and other business purposes. 84 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 10. Mobile app messaging Through the integration of Campaign and Engage, you can segment and upload contact information from Campaign that you can use to create and send mobile push notifications from Engage. The integration provides resources for the following mobile app messaging scenarios. v Upload mobile consent and address data from your marketing databases to Engage to send mobile app messages. See “Uploading mobile push contact data to Engage” on page 86. v Send mobile app messages automatically after you upload information that you select from your marketing databases. See “Sending push notifications after uploading contact data” on page 86. v Download mobile app contact data from Engage to Campaign See “Downloading mobile push contact data” on page 87. v Download mobile push notification tracking data from Engage to Campaign See “Downloading mobile push tracking data” on page 87. Mobile push contact synchronization between Campaign and Engage To support mobile app messaging in Engage with mobile app user information from Campaign, the data must be synchronized between Campaign and Engage. The integration between Campaign and Engage can link mobile app user data through a common identifier. Campaign identifies mobile app users by their audienceID. An audienceID can be any type of identifier that uniquely identifies a particular individual. For example, you might use a customerID as an audienceID. If a mobile app is implemented with Engage, a series of interactions between the mobile app user, Campaign, and Engage establishes a link between Campaign and Engage. The following example assumes that Campaign identifies users through their customerID. v The first time a mobile user installs and runs the mobile app, the mobile userID and channel ID is sent to the IBM Marketing Cloud and Engage. A contact record is created in an Engage database that is enabled for mobile push. v When the mobile user logs in from the mobile app to the client system, the client authentication service takes the following actions: – Authenticate the user – Looks up the customerID to identify the authenticated user. – Stores the user’s mobile userID and channelID with the customerID in the client database. The database is mapped to Campaign as a data source. – The authentication service calls an Engage API to pass the customerID, mobile user ID, and channel ID to Engage. The call associates the Mobile User ID with the customer ID. After Campaign and Engage exchange identifier information between Campaign and Engage, Campaign flowcharts can begin to segment the data. As new user © Copyright IBM Corp. 2014, 2017 85 information becomes available in Campaign, you can use the contactUpload script to update the corresponding mobile app user records in Engage. Uploading mobile push contact data to Engage You can upload contact data for mobile app users from Campaign to Engage to update records for mobile app users. To support mobile push notifications in Engage, the contact data that you upload from Campaign must contain information that corresponds to the mobile push data in Engage. v Typical data in Campaign – Any type of record that serves as the Campaign audience ID to identify the mobile app user – Data that can be matched to Mobile UserID in IBM Marketing Cloud – Data that can be matched to Channel ID in IBM Marketing Cloud – Other personalization fields (optional) To configure a flowchart to upload Campaign data to an Engage database, add a flowchart trigger that contains the contactUpload script to the Mail List process. The script must reference the database ID of a database that is enabled for mobile push. v Trigger design: contactUpload -i <inputFile> -l <databaseID> v Example: <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 When the flowchart run finishes, the trigger runs the script and the system uploads the tab separated input file to the specified Engage database. All matching user records in the database are updated with the uploaded data. New records are added to the database. After you update the Engage database, you can include the records in one or more contact lists. Sending push notifications after uploading contact data You can send mobile push notifications to a selected group of mobile app users immediately after uploading Campaign data to an Engage contact list that defines the group. Create and publish a mobile app template before you run the script or flowchart. For information about how to create a mobile template, go to the IBM Marketing Cloud Knowledge Base and review: Create a Mobile App Template. Add the contactUpload script to a trigger in a Mail List process at the end of the flowchart and specify the mobile template ID as a script parameter. Add the Mail List process to the end of the flowchart. The upstream Campaign processes must provide values that synchronize with Mobile User ID and Channel ID. v Trigger design contactUpload -i <inputFile> -l <databaseID> -p <contactListIDs> -e <templateID> -u <channel> v Example 86 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide <CS_HOME>\bin\contactUpload.bat -i ..\<a folder>\FC_out.tsv -l 12345 -p 98765 -e 45678 -u push When the flowchart run finishes, the system uploads the tab separated file that contains the flowchart output to Engage. Engage processes the new contact list and transmits mobile push notifications to the app users that were included in the contact list. The contents of the message depend on the design of the mobile template that you created in advance. Downloading mobile push contact data Run the contactDownload script to download mobile app contact data from an Engage database or contact list that is enabled for mobile app messaging. You can also download mobile app data that is returned by a query. The ID of the mobile app enabled database, list, or query is a required input to contactDownload. To download all available contact data for the specified database, contact list, or query, run contactDownload from the command line or as a trigger in a Schedule process and specify only a list ID v Trigger design contactDownload -l <listID> v Example <CS_HOME>\bin\contactDownload.bat -l 12345 When you run the script or flowchart, the contactDownload script retrieves the available contact data from Engage and creates the tab-separated download output file in the default location or in the location that you specify with the -o option. You can use various Campaign processes to select the downloaded data and use it in Campaign. Downloading mobile push tracking data You must use the UBX Toolkit to download mobile push tracking data from Engage to Campaign. Before you begin Register Campaign and Engage as UBX endpoints. Register Engage as an event publisher and Campaign as an event consumer. In UBX, subscribe Campaign to the Mobile Push events that Engage publishes. About this task To download mobile push tracking data, you must run the eventsDownload script to download event data from UBX to a to a local directory. You can run the script manually or as a scheduled job. By default, the script places the file in ../AppData/downloads where you installed the UBX Toolkit, but you can specify a different default directory in the config.properties file. The script downloads the available tracking data as a tab-separated file (.tsv). After you download the tracking data to a local directory, run the eventsImport script to add event data from the TSV file to a local SQL database. You can also Chapter 10. Mobile app messaging 87 run this script manually or as a scheduled job. You can insert a single file or a single directory into the database. The UBX Toolkit provides a mapping file to specify how the event data is added to the database. Procedure 1. Run the eventsDownload script manually, or as a scheduled job. eventsDownload [-c <config properties file>] Use the -c option if you use a configuration properties file that is not in the default location (conf directory) or that does not use the default file name (config.properties) Example: eventsDownload.bat [-c <config properties file>] For more information about how to run the script, see the instructions on IBM Knowledge Center for downloading events from UBX at: Downloading events from UBX The UBX toolkit downloads event data from UBX in JSON format. When the event data has finished downloading, the eventsDownload script converts JSON objects to TSV format. 2. Run the eventsImport script. Specify the mapping file that you have created for the UBX Toolkit. eventsImport -m <mapping file> -i <source TSV file> -f <source directory> The default mapping file is EventsToDBMapping.xml. Example: eventsImport.bat -m EventsToDBMapping.xml -i MobilePushSent -f ToolKit_home/AppData/downloads For more information about how to run the script, see the instructions in IBM Knowledge Center at: Importing event data into a database. Results After the eventsImport script imports the event data to the specified database, the script moves the TSV files to the <UBX Toolkit home directory>/AppData/ dataProcessed directory. The dataProcessed folder is created under the downloads folder that you specify in config.properties. What to do next To download tracking data on a recurring basis, you can run the script according to a schedule. Use Campaign flowcharts to select the tracking data for use in Campaign for retargeting and other business purposes. 88 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 11. Reference: Mapping files The IBM Campaign integration with IBM Engage provides sample mapping files that you can use as a models for custom mapping files in your environment. example_mappingFile The example_mappingFile is in the ../<CS_HOME>/conf directory. <!-Licensed Materials - Property of IBM IBM Campaign Materials Configuration file for IBM Campaign integration with IBM Marketing Cloud Engage (c) Copyright IBM Corporation 2014 - 2015. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. --> <!-This is an example mapping file. A mapping file is a parameter to contactUpload and tableUpload. You can assign any name to this file and save it in any location. The parameters given to contactUpload and tableUpload specify the location and name of the file. --> <LIST_IMPORT> <LIST_INFO> <!-- The LIST_INFO section is required. --> <ACTION>ADD_AND_UPDATE</ACTION> <!-- Actions ADD_AND_UPDATE and REPLACE are supported import actions. Use ADD_AND_UPDATE with contactUpload or tableUpload to add new data to a database, contact list, or relational table or to update existing data. Use REPLACE with tableUpload only. Enter REPLACE to remove all data from a relational table and replace it with new data. --> <LIST_VISIBILITY>1</LIST_VISIBILITY> <!-- Do not change this value. --> <!-- LIST_VISIBLIITY must be set to 1, which allows the data to be added to shared databases and relational tables. --> <FILE_TYPE>1</FILE_TYPE> <!-- Do not change this value --> <!-- FILE_TYPE must be set to 1 to allow upload of .tsv files. --> <LIST_ID>Update this value</LIST_ID> <!-- You must update this value. --> <!-- Set LIST_ID to match the id of the database, contact list, or relational table that you are uploading to.--> <LIST_DATE_FORMAT>MM/dd/yyyy</LIST_DATE_FORMAT> <!-- The date format must match the date format that is used in the data that you are uploading. --> <HASHHEADERS>true</HASHHEADERS> <!-- Do not change this value. --> <!-- HASHHEADERS must be set to true, which treats the first row to provide the column names. --> © Copyright IBM Corp. 2014, 2017 89 </LIST_INFO> <CONTACT_LISTS> <!-- The CONTACT_LISTS section is optional. Enter values in this section to upload data to one or more contact lists. --> <!-- Enter the contact list ID (5 digit value)for each contact list that receives data as part of the upload. --> <CONTACT_LIST_ID>update this value</CONTACT_LIST_ID> <CONTACT_LIST_ID>update this value</CONTACT_LIST_ID> <!-- Add more contact list IDs as required. --> </CONTACT_LISTS> <MAPPING> <!-- The MAPPING section is required. Provide one <COLUMN> section for each column in the .tsv file. --> <!-- Within each <COLUMN>, define INDEX, NAME, and INCLUDE. <!-- Define <INDEX> as a value that corresponds to the order of columns in the .tsv file. For example, the first column in the .tsv file matches the first column in the mapping file. Assign 1 as the <INDEX> value. For the second column, assign 2 as the <INDEX> value. Continue for the remaining columns in the .tsv file. --> <!-- Define <NAME> to match the column name of the database in Engage. --> <!-- Define <INCLUDE> as either True or False. True: The column is uploaded to Silverpop. False: The column is not included in the upload to Silverpop --> <!-- Example columns below. Edit to match your data. REQUIRED: The column that provides recipient email addresses must be named "Email". --> <COLUMN> <INDEX>1</INDEX> <NAME>CustID</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>2</INDEX> <NAME>LastName</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>3</INDEX> <NAME>FirstName</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>4</INDEX> <NAME>Email</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>5</INDEX> <NAME>NumericField</NAME> <INCLUDE>true</INCLUDE> </COLUMN> 90 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide <COLUMN> <INDEX>6</INDEX> <NAME>DateField</NAME> <INCLUDE>true</INCLUDE> </COLUMN> </MAPPING> </LIST_IMPORT> example_SMSmappingFile You must configure a mapping file for SMS messaging. The example_SMSmapping file contains tags that are specific to SMS messaging. The file is in is in the ../<CS_HOME>/conf directory. <!-Licensed Materials - Property of IBM IBM Campaign Materials Configuration file for IBM Campaign integration with IBM Marketing Cloud Engage (c) Copyright IBM Corporation 2014 - 2015. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. --> <!-This is an example mapping file for use with SMS messaging. A mapping file is a parameter to the contactUpload script. You can assign any name to this file and save it in any location. The parameters that you give to contactUpload specify the location and name of the file. --> <LIST_IMPORT> <LIST_INFO> <!-- LIST_INFO section is required --> <ACTION>ADD_AND_UPDATE</ACTION> <!-- Actions ADD_AND_UPDATE and OPT_OUT are supported import actions. Use ADD_AND_UPDATE with contactUpload to add new SMS-related data, or update existing data, in an Engage database and one or more Engage contact lists. Use OPT_OUT with contactUpload to update existing contacts who have opted out and should not receive SMS messages. The .tsv file that is uploaded must specify the value of the CONSENT_STATUS_TYPE as "OPTED_OUT". --> <LIST_VISIBILITY>1</LIST_VISIBILITY> <!-- Do not change this value --> <!-- LIST_VISIBLIITY must be set to 1 so that the data can be added to shared databases --> <FILE_TYPE>1</FILE_TYPE> <!-- Do not change this value --> <!-- FILE_TYPE must be set to 1 to allow upload of .tsv files --> <LIST_ID>Update this value</LIST_ID> <!-- You must update this value. --> <!-- Set LIST_ID to match the id of the database that you are uploading to. --> <LIST_DATE_FORMAT>MM/DD/yyyy</LIST_DATE_FORMAT> <!-- The date format must match the date format that is used in the data that you are uploading. --> <HASHEADERS>true</HASHEADERS> Chapter 11. Reference: Mapping files 91 <!-- Do not change this value. --> <!-- HASHHEADERS must be set to true, which treats the first row to provide the column names. --> </LIST_INFO> <CONTACT_LISTS> <!-- The CONTACT_LISTS section is optional. Enter values in this section to upload data to one or more contact lists. --> <!-- Enter the contact list ID (5 digit value)for each contact list that receives data as part of the upload. --> <CONTACT_LIST_ID>Update this value</CONTACT_LIST_ID> <CONTACT_LIST_ID>Update this value</CONTACT_LIST_ID> <!-- Add more contact list IDs as required. --> </CONTACT_LISTS> <CONSENT> <!-- CONSENT section is required --> <CHANNEL>SMS</CHANNEL> <!-- CHANNEL section is required. --> <!-- Do not change this value. --> <TEXT_TO_JOIN_PROGRAM_ID>Update this value</TEXT_TO_JOIN_PROGRAM_ID> <!-- You must update this value. --> <!-- Enter the Text to Join ID (6 digits) for the SMS program.--> <OVERRIDE_ON_NO_CHANGE>true OR false</OVERRIDE_ON_NO_CHANGE> <!-- Define whether or not to update consent information for matched records when you upload new consent data. This setting applies when the import action is ADD_AND_UPDATE. True: Replace existing consent data during data upload. Update Consent Date, Consent Source, and Consent Status. False: Do not change consent data for existing matched records. --> </CONSENT> <SYNC_FIELDS> <!-- SYNC_FIELDS section is required --> <SYNC_FIELD> <NAME>Update this value</NAME> <!-- You must update this value. --> <!-- Specify the field in the .tsv file that provides the SMS phone number. In Engage, this field has the Field Type: SMS Phone Number --> </SYNC_FIELD> </SYNC_FIELDS> <MAPPING> <!-- The MAPPING section is required. Provide one <COLUMN> section for each column in the .tsv file. --> <!-- Within each <COLUMN>, define INDEX, NAME, and INCLUDE. --> <!-- SMS messaging requires recipient consent. To designate fields that provide consent data, define <IS_CONSENT>. You must define a consent field that provides the CONSENT_STATUS_CODE. Optionally, if your database records the date and method of consent, you can define fields 92 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide for CONSENT_DATE and CONSENT_SOURCE. --> <!-- Define <INDEX> as a value that corresponds to the order of columns in the .tsv file. For example, the first column in the .tsv file matches the first column in the mapping file. Assign 1 as the <INDEX> value. For the second column, assign 2 as the <INDEX> value. Continue for the remaining columns in the .tsv file. --> <!-- Define <NAME> to match the column name of the database in Engage. --> <!-- Define <INCLUDE> as either True or False. True: The column is uploaded to Engage. False: The column is not included in the upload to Engage --> <!-- Define <IS_CONSENT> as either True or False. Required for the field that provides CONSENT_STATUS_CODE. True: The field is related to recipient opt in. False: The column is not related to recipient opt in. --> <!-- Example columns below. Edit to match your data. REQUIRED: Although this mapping file is used with SMS text messages, you must define a column in the .tsv file that maps to the column in the Engage database for email addresses. The column must be named "Email". --> <COLUMN> <INDEX>1</INDEX> <IS_CONSENT>true</IS_CONSENT> <!--Required --> <NAME>CONSENT_STATUS_CODE</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>2</INDEX> <IS_CONSENT>true</IS_CONSENT> <!--Optional --> <NAME>CONSENT_SOURCE</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>3</INDEX> <IS_CONSENT>true</IS_CONSENT> <!--Optional --> <NAME>CONSENT_DATE</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <COLUMN> <INDEX>4</INDEX> <NAME>Email</NAME> <INCLUDE>true</INCLUDE> </COLUMN> <!--Required --> <COLUMN> <INDEX>5</INDEX> <NAME>Enter a Name</NAME> <INCLUDE>true or false</INCLUDE> </COLUMN> <COLUMN> <INDEX>6</INDEX> <NAME>Enter a name</NAME> <INCLUDE>true or false</INCLUDE> </COLUMN> <COLUMN> <INDEX>7</INDEX> Chapter 11. Reference: Mapping files 93 <NAME>Enter a name</NAME> <INCLUDE>true or false</INCLUDE> </COLUMN> <!-- Define additional columns as needed. --> </MAPPING> </LIST_IMPORT> Related concepts: “Data required for SMS consent” on page 78 “SMS Opt-in and Opt-out synchronization between Campaign and Engage” on page 82 94 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Chapter 12. Custom tracking tables The Campaign and Engage integration package includes several DDL scripts that you can use to create database tables to receive email contact and tracking data that you download from Engage. You create the tables in the Campaign schema. The Campaign and Engage integration recognizes several different types of message responses. In the tracking tables, the responses are represented as different types of contact events. Each table contains information to describe different aspects of the message contact and tracking data that Engage provides. The type of contact event determines the type of data that a tracking table contains. Table Description More detail SP_Attachment Records when an attachment is downloaded or viewed. “SP_Attachment” on page 97 SP_BounceReply Records of unsuccessful message delivery. “SP_BounceReply” on page 98 SP_Click Records for link clicks in messages. “SP_Click” on page 98 SP_Conversion Records of Engage conversion events. “SP_Conversion” on page 99 SP_Forward Records of messages that were forwarded. “SP_Forward” on page 100 SP_Open Records of message opens. “SP_Open” on page 101 SP_OptIn Records that are marked as opted-in. SP_OptOut Records that are marked as opted-out. “SP_OptIn” on page 102 SP_Sent Records of messages sent. “SP_OptOut” on page 103 SP_Suppressed Records of recipients for whom messages are suppressed. “SP_Suppressed” on page 104 SP_UploadAudit Record of all data uploads and related API calls. “SP_UploadAudit” on page 105 Event types The tracking tables provide data to describe different types of message responses. The type of response is considered an event type. The tracking tables include values for the following event types. Event type Valid value Open 0 Click Through 1 Clickstream 2 Conversion 3 Attachment 4 © Copyright IBM Corp. 2014, 2017 95 Event type Valid value Media 5 Forward 6 Opt In 7 Opt Out 8 Reply Abuse 10 Reply Change Address 11 Reply Mail Block 12 Reply Mail Restriction 13 Reply Other 14 Suppressed 15 Sent 16 Soft Bounce 98 Hard Bounce 99 Report IDs If you have configured reporting in Engage, the downloaded data includes a Report ID. If you have not configured reporting for Engage mailings, Report IDs do not appear in the tracking tables. Report IDs are assigned in various ways, depending on the type of mailing. v For event-driven Autoresponders, a single Report ID is associated with every mailing for a day. v For a recurring Automated Message, a single Report ID is associated with each occurrence of the mailing. v For a standard mailing, there is a one-to-one relationship between a Report ID and Mailing ID. Reasons for contact suppression Engage sometimes does not send a message to an address for various reasons. If Engage suppresses a message, the reason for doing so is included in the data that you downloaded from Engage. Engage provides the following reasons for contact suppression. 96 Suppression reason Valid values Invalid System Email Domain 1 Invalid System Email Local 2 Invalid Organization Email Domain 3 Organization Suppression List 4 Global Suppression 5 Invalid Organization Email Local 6 Frequency Control 7 Database Level Suppression 8 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Suppression reason Valid values Query Level Suppression 9 Mailing Level Suppression 10 SP_Attachment The SP_Attachment table contains data that Engage collects when a contact downloads an attachment or views an attached video. Column Description Valid values Data type RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Primary Key Yes nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. bigint email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime ContentID The identifier that the user specified for the attachment. nvarchar MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar Chapter 12. Custom tracking tables 97 SP_BounceReply The SP_BounceReply table contains data that Engage collects to record when messages are not delivered successfully. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Data type Primary Key Yes nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_Click The SP_Click table contains data that Engage collects to record when message recipients click links in the messages that they receive. Column Description RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint 98 Valid values IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Data type Primary Key Yes Column Description Valid values Data type RecipientType The type of message contact. 0 Regular nvarchar Primary Key 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime Body Type The format of the message body that the 0 HTML contact received. 1 AOL nvarchar 2 TEXT 3 WEB (click-to-view) Click Name The name that the user specified for the link or Clickstream. URL The URL that is defined for a Clickthrough or Clickstream. MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar URL nvarchar SP_Conversion The SP_Conversion table contains data that Engage collects to record conversion events. Column Description Valid values Data type RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint Primary Key Yes Chapter 12. Custom tracking tables 99 Column Description Valid values Data type RecipientType The type of message contact. 0 Regular nvarchar Primary Key 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime ConversionAction The action that the user specified for a conversion. nvarchar ConversionDetail The description that the user provided for a conversion. nvarchar MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_Forward The SP_Forward table contains data that Engage collects to record when recipients forwarded the messages that they received. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular 1 Forward 3 Seed 4 Inbox Monitoring 100 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Data type nvarchar Primary Key Yes Column Description Valid values Data type MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint Primary Key See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_Open The SP_Open table contains data that Engage collects to record when recipients open the messages that they received. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Data type Primary Key Yes nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Chapter 12. Custom tracking tables 101 Column Description Valid values Data type Email The email address of the contact. email address nvarchar EventType The type of contact event. Primary Key nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime Body Type The format of the message body that the 0 HTML contact received. 1 AOL nvarchar 2 TEXT 3 WEB (click-to-view) MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_OptIn The SP_OptIn table contains data that Engage collects to indicate that a message recipient has opted in. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Data type nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address “Event types” on page 95 102 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide nvarchar nvarchar Primary Key Yes Column Description Valid values Data type EventTimeStamp The date and time when Engage recorded the contact event. datetime MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar Primary Key SP_OptOut The SP_OptOut table contains data that Engage collects to indicate that a message recipient has opted out. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Data type Primary Key Yes nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar Chapter 12. Custom tracking tables 103 SP_Sent The SP_Sent table contains data that Engage collects to record when it sent messages. Column Description Valid values RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint RecipientType The type of message contact. 0 Regular Data type Primary Key Yes nvarchar 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_Suppressed The SP_Suppressed table contains data that Engage collects to record when it suppresses message transmission. Column Description RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RecipientID The ID of the contact that is associated with the event. bigint 104 Valid values IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Data type Primary Key Yes Column Description Valid values Data type RecipientType The type of message contact. 0 Regular nvarchar Primary Key 1 Forward 3 Seed 4 Inbox Monitoring MailingID The ID of the Sent Mailing that is associated with the event. bigint ReportID If reporting is configured in Engage, the ID of the report that is associated with the mailing. If reporting is not configured, this field is empty. bigint See “Report IDs” on page 96 CampaignID The ID of the Group of Automated Messages that is associated with the contact event. Email The email address of the contact. EventType The type of contact event. email address nvarchar nvarchar “Event types” on page 95 EventTimeStamp The date and time when Engage recorded the contact event. datetime SuppressionReason The reason a contact was suppressed and a message was not sent. nvarchar “Reasons for contact suppression” on page 96 MailingName The name of the mailing, as it is defined in Engage. Includes an additional identifier as a prefix, if defined during data upload. nvarchar SP_UploadAudit The SP_UploadAudit table contains data for a record of all data uploads from Campaign and related API calls. The system updates the table as it completes each step in the upload process. Column Description Valid values Data type RecordID A unique value to identify the contact record. Assigned when the record is added to the table. bigint RunID The ID of the script execution. It is used to group all steps that are related to a specific upload run. bigint UserName The name of the SFTP user that performs the upload. nvarchar Primary Key Yes Chapter 12. Custom tracking tables 105 Column Description Valid values Data type ScriptName Name of the upload script. tableUpload nvarchar listUpload Step The upload step that is being performed upload by the script (upload or API name) <API name> nvarchar FileName Name of the file being processed in the step (as applicable). nvarchar SPListID List or Table ID in IBM Marketing Cloud (when applicable) nvarchar SPMailingID Mailing ID in IBM Marketing Cloud nvarchar StartTime The time the step started. datetime LastStatusUpdate The time of the most recent status update. datetime Status Current status of the upload. STARTED nvarchar INPROGRESS FAILED SUCCESS ErrorMessage 106 Error message if the upload fails. Populated for FAILED status only. IBM Campaign Version-independent Integration with IBM Engage: Integration Guide nvarchar Primary Key Before you contact IBM technical support If you encounter a problem that you cannot resolve by consulting the documentation, your company's designated support contact can log a call with IBM technical support. Use these guidelines to ensure that your problem is resolved efficiently and successfully. If you are not a designated support contact at your company, contact your IBM administrator for information. Note: Technical Support does not write or create API scripts. For assistance in implementing our API offerings, contact IBM Professional Services. Information to gather Before you contact IBM technical support, gather the following information: v A brief description of the nature of your issue. v Detailed error messages that you see when the issue occurs. v Detailed steps to reproduce the issue. v Related log files, session files, configuration files, and data files. v Information about your product and system environment, which you can obtain as described in "System information." System information When you call IBM technical support, you might be asked to provide information about your environment. If your problem does not prevent you from logging in, much of this information is available on the About page, which provides information about your installed IBM applications. You can access the About page by selecting Help > About. If the About page is not accessible, check for a version.txt file that is located under the installation directory for your application. Contact information for IBM technical support For ways to contact IBM technical support, see the IBM Product Technical Support website: (http://www.ibm.com/support/entry/portal/open_service_request). Note: To enter a support request, you must log in with an IBM account. This account must be linked to your IBM customer number. To learn more about associating your account with your IBM customer number, see Support Resources > Entitled Software Support on the Support Portal. © Copyright IBM Corp. 2014, 2017 107 108 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: Intellectual Property Licensing Legal and Intellectual Property Law IBM Japan, Ltd. 19-21, Nihonbashi-Hakozakicho, Chuo-ku Tokyo 103-8510, Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. © Copyright IBM Corp. 2014, 2017 109 IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Corporation B1WA LKG1 550 King Street Littleton, MA 01460-1250 U.S.A. Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only. All IBM prices shown are IBM's suggested retail prices, are current and are subject to change without notice. Dealer prices may vary. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating 110 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided "AS IS", without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs. If you are viewing this information softcopy, the photographs and color illustrations may not appear. Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml. Privacy Policy and Terms of Use Considerations IBM Software products, including software as a service solutions, ("Software Offerings") may use cookies or other technologies to collect product usage information, to help improve the end user experience, to tailor interactions with the end user or for other purposes. A cookie is a piece of data that a web site can send to your browser, which may then be stored on your computer as a tag that identifies your computer. In many cases, no personal information is collected by these cookies. If a Software Offering you are using enables you to collect personal information through cookies and similar technologies, we inform you about the specifics below. Depending upon the configurations deployed, this Software Offering may use session and persistent cookies that collect each user's user name, and other personal information for purposes of session management, enhanced user usability, or other usage tracking or functional purposes. These cookies can be disabled, but disabling them will also eliminate the functionality they enable. Various jurisdictions regulate the collection of personal information through cookies and similar technologies. If the configurations deployed for this Software Offering provide you as customer the ability to collect personal information from end users via cookies and other technologies, you should seek your own legal advice about any laws applicable to such data collection, including any requirements for providing notice and consent where appropriate. IBM requires that Clients (1) provide a clear and conspicuous link to Customer's website terms of use (e.g. privacy policy) which includes a link to IBM's and Client's data collection and use practices, (2) notify that cookies and clear gifs/web beacons are being placed on the visitor's computer by IBM on the Client's behalf along with an explanation of the purpose of such technology, and (3) to the extent required by law, obtain consent from website visitors prior to the placement of cookies and clear gifs/web beacons placed by Client or IBM on Client's behalf on website visitor's devices For more information about the use of various technologies, including cookies, for these purposes, See IBM's Online Privacy Statement at: http://www.ibm.com/ privacy/details/us/en section entitled "Cookies, Web Beacons and Other Technologies." Notices 111 112 IBM Campaign Version-independent Integration with IBM Engage: Integration Guide IBM® Printed in USA