Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Teradata FastLoad Reference - Teradata
Document related concepts
Open Database Connectivity wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Clusterpoint wikipedia , lookup
Functional Database Model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Transcript
Teradata FastLoad Reference Release 14.10 B035-2411-082K March 2013 The product or products described in this book are licensed products of Teradata Corporation or its affiliates. Teradata, Active Enterprise Intelligence, Applications-Within, Aprimo, Aprimo Marketing Studio, Aster, BYNET, Claraview, DecisionCast, Gridscale, MyCommerce, Raising Intelligence, Smarter. Faster. Wins., SQL-MapReduce, Teradata Decision Experts, "Teradata Labs" logo, "Teradata Raising Intelligence" logo, Teradata ServiceConnect, Teradata Source Experts, "Teradata The Best Decision Possible" logo, The Best Decision Possible, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the United States and other countries. Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc. AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc. Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access, Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum Support are servicemarks of Axeda Corporation. Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation. GoldenGate is a trademark of Oracle. Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company. Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other countries. Intel, Pentium, and XEON are registered trademarks of Intel Corporation. IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation. Linux is a registered trademark of Linus Torvalds. LSI is a registered trademark of LSI Corporation. Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries. NetVault is a trademark or registered trademark of Quest Software, Inc. in the United States and/or other countries. Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries. Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates. QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation. Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license. SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc. SPARC is a registered trademark of SPARC International, Inc. Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and other countries. Unicode is a registered trademark of Unicode, Inc. in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Other product and company names mentioned herein may be the trademarks of their respective owners. THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN "AS-IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country. Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice. To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please email: [email protected]. Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback. Copyright © 1998-2013 by Teradata Corporation. All Rights Reserved. Preface Purpose This book provides information about Teradata FastLoad (FastLoad), which is a Teradata® Tools and Utilities product. Teradata Tools and Utilities is a group of products designed to work with Teradata Database. Teradata FastLoad is a command-driven utility that quickly loads large amounts of data to empty tables in a Teradata Database. FastLoad uses multiple sessions to load data; however, it loads data into only one table on a Teradata Database per job. Audience This book is intended for use by: • System and application programmers • System administrators Supported Releases This book supports the following releases: • Teradata Database 14.10 • Teradata Tools and Utilities 14.10 • Teradata FastLoad 14.10 Note: See “SHOW VERSIONS” on page 165 to verify the Teradata FastLoad version number. For the most current information about supported releases, do the following: 1 Go to http://www.info.teradata.com/. 2 Click General Search under Online Publications. 3 Type 3119 in the Publication Product ID box. 4 Under Sort By, select Date. 5 Click Search. 6 Open the version of the Teradata Tools and Utilities ##.# Supported Platforms and Product Versions (B035-3119) spreadsheet associated with this release. Teradata FastLoad Reference 3 Preface Prerequisites The spreadsheet includes supported Teradata Database versions, platforms, and product release numbers. Prerequisites The following prerequisite knowledge is required for this product: • Familiarity with computer technology, relational database management systems, and utilities that load and retrieve data • Teradata SQL • Teradata Database Changes to This Book The following changes were made to this book in support of the current release. Changes are marked with change bars. For a complete list of changes to the product, see the Teradata Tools and Utilities Release Definition (B035-2029) associated with this release. Date and Release Description March 2013 • All instances of “channel-attached” were changed to “mainframe-attached.” • Teradata FastLoad supports VARTEXT on z/OS. See “SET RECORD” on page 154. • Teradata FastLoad supports temporal syntaxes prefixed in DELETE/DEL statements. • Text in “Mainframe-Attached Runtime Parameters” on page 35 was corrected. • Teradata FastLoad supports enhanced statement status (8-byte activity count). • The SET RECORD VARTEXT command line parameter should start with a .(dot). See “Syntax Notes” on page 87. Teradata Tools and Utilities 14.10 • Two limitations for loading rows into normalized tables were documented in “Restrictions and Limitations” on page 62. • FastLoad provides an user option to retry on CLI 207 during logon. See “Programming Considerations” on page 52. • Teradata FastLoad no longer has a problem with the INMOD command when the PATH has a space. See “DEFINE” on page 97. • Teradata FastLoad now uses DBS UDFs that will return a list of retryable and restartable errors. • Added Chapter 4: “Troubleshooting.” 4 Teradata FastLoad Reference Preface Additional Information Date and Release Description • If multi-byte characters are used in object names in a Teradata FastLoad script, they must be enclosed in double quotes. • Teradata FastLoad Notify Exits now support DBS Extended Object Name (EON) functionality. See “Teradata FastLoad/Notify Exit Routine Interface” on page 68 and Appendix C: “INMOD and Notify Exit Routine Examples.” • If the length of the data item name is greater than 120 characters, the DBS will truncate the data item name to the length of 120 characters. • References to Replication Services updated in Glossary. • FastLoad supports the new DBS 14.10 "HELP TABLE" and "HELP SESSION" logic for Extended Object Names. Additional Information Additional information that supports this product and Teradata Tools and Utilities is available at the web sites listed in the following table. Type of Information Description Access to Information Release overview Use the Release Definition for the following information: 1 Go to http://www.info.teradata.com/. • Overview of all of the products in the release • Information received too late to be included in the manuals • Operating systems and Teradata Database versions that are certified to work with each product • Version numbers of each product and the documentation for each product • Information about available training and the support center 3 Type 2029 in the Publication Product ID box. Late information Teradata FastLoad Reference 2 Click General Search under Online Publications. 4 Click Search. 5 Select the appropriate Release Definition from the search results. 5 Preface Additional Information Type of Information Description Access to Information Additional product information Use the Teradata Information Products web site to view or download specific manuals that supply related or additional information to this manual. 1 Go to http://www.info.teradata.com/. 2 Click Data Warehousing under Online Publications, Browse by Category. 3 Do one of the following: • For a list of Teradata Tools and Utilities documents, click Teradata Tools and Utilities, and then select an item under Releases or Products. • Select a link to any of the data warehousing publications categories listed. Specific books related to Teradata FastLoad are as follows: • Introduction to Teradata (B035-1091) • Database Administration (B035-1093) • Database Design (B035-1094) • Messages (B035-1096) • Security Administration (B035-1100) • Utilities (B035-1102) • International Character Set Support (B035-1132) • SQL Fundamentals (B035-1141) • SQL Data Types and Literals (B035-1143) • SQL Data Definition Language (B035-1144) • SQL Data Manipulation Language (B035-1146) • SQL External Routine Programming (B035-1147) • Teradata Tools and Utilities Command Summary (B035-2401) • Basic Teradata Query Reference (B035-2414) • Teradata Call-Level Interface Version 2 Reference for Mainframe-Attached Systems (B035-2417) • Teradata Call-Level Interface Version 2 Reference for Network-Attached Systems (B035-2418) • Teradata Tools and Utilities Access Module Programmer Guide (B035-2424) • Teradata Tools and Utilities Access Module Reference (B035-2425) • Teradata Dynamic Workload Manager User Guide (B035-2513) CD-ROM images 6 Access a link to a downloadable CD-ROM image of all customer documentation for this release. Customers are authorized to create CD-ROMs for their use from this image. 1 Go to http://www.info.teradata.com/. 2 Click Data Warehousing under Online Publications, Browse by Category. 3 Click CD-ROM Images. Teradata FastLoad Reference Preface Additional Information Type of Information Description Access to Information Ordering information for manuals Use the Teradata Information Products web site to order printed versions of manuals. 1 Go to http://www.info.teradata.com/. 2 Click How to Order under Print & CD Publications. 3 Follow the ordering instructions. General information about Teradata The Teradata home page provides links to numerous sources of information about Teradata. Links include: 1 Go to Teradata.com. 2 Select a link. • Executive reports, case studies of customer experiences with Teradata, and thought leadership • Technical information, solutions, and expert advice • Press releases, mentions, and media resources Teradata FastLoad Reference 7 Preface Additional Information 8 Teradata FastLoad Reference Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Supported Releases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 Changes to This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Chapter 1: Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Teradata FastLoad Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What It Does. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 17 18 18 Operating Features and Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Operating Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Data Transfer Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Data Conversion Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Input Data Formats. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Formatted Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unformatted Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Binary Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Text Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Variable-length Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unsupported Data Sets and Tapes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 21 22 22 23 24 24 Teradata FastLoad Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Teradata FastLoad Command Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Teradata SQL Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Teradata FastLoad Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Teradata FastLoad Reference 9 Table of Contents Chapter 2: Using Teradata FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 Invoking Teradata FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 File Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 Interactive Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34 Batch Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34 Mainframe-Attached Runtime Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35 Network-Attached Runtime Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39 z/OS Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46 UNIX and Windows Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 Terminating Teradata FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 Normal Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 Abort Termination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 Restart a Paused Teradata FastLoad Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 After a Client System or Teradata FastLoad Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 After a Database Overfill Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 After a Teradata Database Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 After an Unrecoverable Error Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 Restart Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51 Programming Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52 Teradata FastLoad Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52 Character Set Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 Unicode® Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57 ANSI/SQL DateTime Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57 Checkpoint Tradeoffs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58 Checkpoint Support with Access Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58 Concurrent Load Utility Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59 Data Conversion Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59 Error Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Nonunique Index Sorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Duplicate Rows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Range Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Record Mode Load Anomaly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61 UNIX Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61 Restrictions and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62 Loading No Primary Index Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63 Conventions for Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63 INMOD and Notify Exit Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64 10 Teradata FastLoad Reference Table of Contents Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Programming Considerations for Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Programming Structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Addressing Mode on z/OS Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teradata FastLoad/INMOD Routine Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teradata FastLoad/Notify Exit Routine Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teradata FastLoad Sample INMOD Routines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating Custom INMOD Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 64 65 66 66 68 70 72 Teradata FastLoad Job Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enter Teradata FastLoad Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teradata FastLoad Job Script Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 73 73 73 Run Multifile Teradata FastLoad Jobs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Initiate the Teradata FastLoad Job and Loading the First Data Source . . . . . . . . . . . . . . Restart the Teradata FastLoad Job and Loading the Second Data Source . . . . . . . . . . . . Restart the Teradata FastLoad Job and Loading the Third Data Source. . . . . . . . . . . . . . Terminate the Teradata FastLoad Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Command Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 75 76 76 76 76 Handling Teradata FastLoad Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Teradata FastLoad Error Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error Table Formats. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Correcting Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Procedure for Correcting Errors in the First Error Table. . . . . . . . . . . . . . . . . . . . . . . . . . Procedure for Correcting Errors in the Second Error Table . . . . . . . . . . . . . . . . . . . . . . . 78 78 79 79 80 80 82 Handling RDBMS Retryable and Restartable Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 FastLoad TMSM Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 FastLoad Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Performance Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Chapter 3: Teradata FastLoad Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Syntax Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Object Name Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Geospatial Data Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LOB Data Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NUMBER Data Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 87 88 88 88 AXSMOD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 BEGIN LOADING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Teradata FastLoad Reference 11 Table of Contents CLEAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95 DATEFORM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96 DEFINE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97 END LOADING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117 ERRLIMIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .119 HELP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121 HELP TABLE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 INSERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .125 LOGDATA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .129 LOGMECH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .130 LOGOFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .131 LOGON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133 NOTIFY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137 OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .142 QUIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145 Completion Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146 RECORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147 Completion Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .148 Completion Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .148 RUN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149 SESSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151 SET RECORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154 SET SESSION CHARSET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .161 SHOW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 SHOW VERSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .165 SLEEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .167 TENACITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170 Chapter 4: Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 Compiler Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 Appendix A: How to Read Syntax Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175 Syntax Diagram Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175 12 Teradata FastLoad Reference Table of Contents Appendix B: Multifile Teradata FastLoad Job Script Examples . . . . . . . . . . . . . 181 First Output File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Second Output File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Third Output File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Appendix C: INMOD and Notify Exit Routine Examples . . . . . . . . . . . . . . . . . . . . . . 191 z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assembler INMOD Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . COBOL INMOD Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PL/I INMOD Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM C INMOD Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 191 194 196 197 UNIX OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 BLKEXIT.C Sample INMOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 BLKEXITR.C Sample INMOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 All Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Notify Exit Routine Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assembler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . COBOL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PL/I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 217 219 220 222 IBM C or C++ INMODs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 UNIX OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SPARC Systems Running Oracle Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Opteron Systems Running Oracle Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HP-UX PA RISC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HP-UX Itanium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . z/Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 225 225 226 227 228 229 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 Teradata FastLoad Reference 13 Table of Contents Appendix E: User-Defined-Types and User-Defined-Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231 User-Defined-Types and User-Defined-Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231 User-Defined-Types (UDTs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231 User-Defined-Methods (UDMs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232 Creating UDTs with FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232 Inserting and Retrieving UDTs with Client Products. . . . . . . . . . . . . . . . . . . . . . . . . . . . .232 External Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232 Inserting UDTs with FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233 Retrieving UDTs with FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233 Retrieving UDT Metadata with FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .235 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .251 14 Teradata FastLoad Reference List of Tables Table 1: Formatted Record Field Descriptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Table 2: Unformatted Record Field Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Table 3: Binary Record Field Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Table 4: Teradata FastLoad Commands for Session Control . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Table 5: Teradata FastLoad Commands for Data Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Table 6: Teradata SQL Statements Supported in Teradata FastLoad. . . . . . . . . . . . . . . . . . . . 26 Table 7: FastLoad Data Sets/Files and Input/Output Devices. . . . . . . . . . . . . . . . . . . . . . . . . . 33 Table 8: Runtime Parameters (Mainframe-Attached Systems) . . . . . . . . . . . . . . . . . . . . . . . . 36 Table 9: Runtime Parameters (Network-Attached Systems) . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Table 10: Character Sets Supported by Teradata FastLoad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Table 11: Site-Defined Character Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Table 12: Range Constraint Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Table 13: Programming Languages Supported by Platform and Type of User-Developed Routine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Table 14: Programming Structures for Teradata FastLoad/INMOD Communication . . . . . 65 Table 15: Entry Points for INMOD Routines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Table 16: Teradata FastLoad--to--INMOD Interface Status Codes . . . . . . . . . . . . . . . . . . . . . 66 Table 17: INMOD-to-Teradata FastLoad Interface Status Codes . . . . . . . . . . . . . . . . . . . . . . 67 Table 18: Events Passed to the Notify Exit Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Table 19: Sample INMOD Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Table 20: FastLoad Entering Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Table 21: Teradata FastLoad Terminating Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Table 22: Error Conditions that Cause Teradata Database to Reject an Input Data Record . 78 Table 23: Error Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Table 24: errortname1 Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Table 25: Usage Notes for AXSMOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Table 26: Usage Note for BEGIN LOADING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Table 27: Usage Notes for DATEFORM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Table 28: Input Length and Field Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Table 29: Limitations by Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Table 30: ANSI/SQL DateTime Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Table 31: Usage Notes for END LOADING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Teradata FastLoad Reference 15 List of Tables Table 32: RECORD Completion Message Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118 Table 33: Usage Notes for ERRLIMIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .119 Table 34: Usage Notes for HELP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121 Table 35: Usage Notes for HELP TABLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 Table 36: Usage Notes for INSERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126 Table 37: LOGOFF Command Usage Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .131 Table 38: LOGOFF Command Completion Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .132 Table 39: Usage Notes for LOGON. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135 Table 40: Events that Create Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139 Table 41: Usage Notes for NOTIFY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139 Table 42: NOTIFY Command Message Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140 Table 43: OS Command Examples on a UNIX Client System . . . . . . . . . . . . . . . . . . . . . . . . .142 Table 44: Example OS Command on Windows Client System . . . . . . . . . . . . . . . . . . . . . . . .143 Table 45: Example Procedure for OS Command on z/OS Client System . . . . . . . . . . . . . . . .144 Table 46: Usage Notes for QUIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145 Table 47: Usage Notes for RECORD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147 Table 48: Usage Notes for Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149 Table 49: Usage Notes for SESSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152 Table 50: Usage Notes for SET SESSION CHARSET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .161 Table 51: Usage Notes for SLEEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .168 Table 52: Usage Notes for TENACITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171 Table 53: Compiler Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 16 Teradata FastLoad Reference CHAPTER 1 Overview This chapter provides an introductory overview of the Teradata FastLoad utility. Topics include: • Teradata FastLoad Utility • Operating Features and Capabilities • Input Data Formats • Teradata FastLoad Commands • Teradata FastLoad Example Teradata FastLoad Utility This section describes Teradata FastLoad features and operation. Description Teradata FastLoad is a command-driven utility which can be used to quickly load large amounts of data in an empty table on a Teradata Database. Data can be loaded from: • Disk or tape files on a mainframe-attached client system • Input files on a network-attached workstation • Special input module (INMOD) routines written to select, validate, and preprocess input data • Any other device providing properly formatted source data Teradata FastLoad uses multiple sessions to load data. However, it loads data into only one table on a Teradata Database per job. To load data into more than one table in the Teradata Database, multiple Teradata FastLoad jobs must be submitted, one for each table. Note: Full tape support is not available for any function in Teradata FastLoad for network-attached client systems. To import data from a tape, write a custom access module that interfaces with the tape device. For information about how to write a custom access module, refer to the Teradata Tools and Utilities Access Module Programmer Guide (B035-2424). Teradata FastLoad Reference 17 Chapter 1: Overview Operating Features and Capabilities What It Does When Teradata FastLoad is invoked, the utility executes the Teradata FastLoad commands and Teradata SQL statements in the Teradata FastLoad job script. These direct Teradata FastLoad to: 1 Log on to the Teradata Database for a specified number of sessions, using username, password, and tdpid/acctid information. 2 Load the input data into the Teradata FastLoad table on the Teradata Database. 3 Log off from the Teradata Database. 4 If the load operation was successful, return the following information about the Teradata FastLoad operation and then terminate: • Total number of records read, skipped, and sent to the Teradata Database • Number of errors posted to the Teradata FastLoad error tables • Number of inserts applied • Number of duplicate rows How It Works Teradata FastLoad processes a series of Teradata FastLoad commands and Teradata SQL statements entered either interactively or in batch mode. Use the Teradata FastLoad commands for session control and data handling of the data transfers. The Teradata SQL statements create, maintain, and drop tables on the Teradata Database. During a load operation, Teradata FastLoad inserts the data from each record of the data source into one row of the table on a Teradata Database. The table on the Teradata Database receiving the data must be empty and have no defined secondary indexes. Note: Teradata FastLoad does not load duplicate rows from the data source to the Teradata Database. (A duplicate row is one in which every field contains the exact same data as the fields of an existing row.) This is true even for MULTISET tables. To load duplicate rows in a MULTISET table, use MultiLoad. Operating Features and Capabilities This section describes the operating modes and character sets for running Teradata FastLoad. For specific information on supported operating systems, refer to Teradata Tools and Utilities ##.##.## Supported and Certified Versions (B035-3119). This spreadsheet shows version numbers and platform information for all Teradata Tools and Utilities products and is available at http://www.info.teradata.com/. Caution: 18 Some system configurations running Teradata FastLoad on a network-attached workstation that has a UNIX® operating system with the data set mounted on a Network File System (NFS) can produce errors and failed or incomplete data transfer operations. Do not use Teradata FastLoad with an NFS-mounted data set. Teradata FastLoad Reference Chapter 1: Overview Data Transfer Capabilities Operating Modes Teradata FastLoad runs in the following operating modes: • Interactive • Batch Character Sets Teradata FastLoad supports Latin, Chinese, Japanese, and Korean character sets for network-attached and mainframe-attached configurations. For additional information about character-set support and definition, speed “Character Set Specification” on page 55. Teradata FastLoad also supports UTF-16 (workstation only) and UTF-8 character sets. For additional information, see “Unicode® Character Sets” on page 57. Note: FastLoad supports 8-byte row counters. All the numbers stated above are of type 8-byte unsigned integer. A FastLoad job can load up to 18,446,744,073,709,551,615 rows. Note: FastLoad supports all Unicode characters in object names. Note: If multi-byte characters are used in object names in a Teradata FastLoad script, they must be enclosed in double quotes. Data Transfer Capabilities This section covers Teradata FastLoad’s data transfer capabilities. On network-attached workstations, Teradata FastLoad uses the TCP/IP network protocol for all data transfer operations. On mainframe-attached systems, Teradata FastLoad transfers data as either: • A multi-volume data set or file • A number of single-volume data sets or files in separate Teradata FastLoad jobs Serial Teradata FastLoad operations can be restarted by loading the next tape in a series instead of beginning with the first tape in a set. In either case, Teradata FastLoad: • Uses multiple Teradata sessions, at one session per AMP, to transfer data • Transfers multiple rows of data within a single message Also, in either case, until the Teradata FastLoad job is completed and the data loaded into the Teradata FastLoad table: • There is no journaling or fallback data • The secondary indexes cannot be defined Teradata FastLoad Reference 19 Chapter 1: Overview Input Data Formats Data Conversion Capabilities Teradata FastLoad can redefine the data type specification of numeric, character, and date input data so it matches the type specification of its destination column in the Teradata FastLoad table on the Teradata Database. If, for example, an input field with numeric type data is targeted for a column with a character data type specification, Teradata FastLoad can change the input data specification to character before inserting it into the table. The types of data conversions which can be specified are: • Numeric-to-numeric (for example integer-to-decimal) • Character-to-numeric • Character-to-date • Date-to-character Note: Redundant conversions, like integer-to-integer, are legal and necessary to support the zoned decimal format. For more information about the zoned decimal format, see Database Design (B035-1094) and SQL Data Types and Literals (B035-1143). Use the datatype specification of the DEFINE command to convert input data to a different type before inserting it into the Teradata FastLoad table on the Teradata Database. For details on data types and data conversions, see SQL Data Types and Literals (B035-1143). Checkpoints Checkpoints are entries posted to a restart log table at regular intervals during the Teradata FastLoad data transfer operation. If processing stops while a Teradata FastLoad job is running, the job can be restarted at the most recent checkpoint. If, for example, 1,000,000 records are being loaded into a table and they have specified checkpoints every 50,000 records, Teradata FastLoad pauses and posts an entry to the restart log table whenever multiples of 50,000 records have been successfully sent to the Teradata Database. If the job stops after record 60,000 has been loaded, the job can be restarted at the record immediately following the last checkpoint, record 50,001. Enable the checkpoint function by specifying a checkpoint value in the BEGIN LOADING command. Input Data Formats This section covers the data formats supported by Teradata FastLoad as well as those data sets and tapes not supported by the utility. Input data is the raw data that Teradata FastLoad loads from the client system or workstation to the Teradata Database. The input data can be either formatted, unformatted, variable-length text or of specific file types, depending on the configuration of the client system. For network-attached configurations, Teradata FastLoad supports the following data formats: 20 Teradata FastLoad Reference Chapter 1: Overview Input Data Formats • Formatted • Unformatted • Binary • Text • Variable-length text For mainframe-attached configurations, Teradata FastLoad supports data sets and tape files with the following record format (RECFM) attributes: • Data sets and tape files with the following record format (RECFM) attributes: • F (fixed) • FB (fixed block) • V (variable) • VB (variable block) • VBS (variable block, spanned) Formatted Data Formatted data on network-attached systems is input data that conforms to the format of data from a Teradata Database source, such as a BTEQ EXPORT file. Each record has: • A two-byte data length field • Optionally, a variable-length indicator bytes field • A variable-length input data field • A one-byte end-of-record delimiter field Table 1 lists the formatted record field descriptions. Table 1: Formatted Record Field Descriptions Input Record Field Description Data length A two-byte field indicating the total length of the record, in bytes, excluding the first two bytes (this field), and the last byte (the end-of-record field) The data length must be specified as an explicit value, not an ASCII value. For a data length of one byte, for example, the specification must be hexadecimal 01. It cannot be hexadecimal 31, or the ASCII equivalent of the number 1. Indicator bytes Optional bytes to indicate null data Input data The actual input data for rows in the Teradata FastLoad table. The input data stream always starts at either: • The third byte of the record, if there are no optional indicator bytes • Immediately after the last optional indicator byte The length of the input data field, less any optional indicator bytes, is as specified in the first two bytes. Teradata FastLoad Reference 21 Chapter 1: Overview Input Data Formats Table 1: Formatted Record Field Descriptions Input Record Field Description End-of-record The end-of-record delimiter field can be either of two hexadecimal values: • 0A (line feed) • 0D (carriage return) Unformatted Data Unformatted data on network-attached systems is data that does not conform to the format of data from a Teradata Database source. It may, however, originate from a Teradata Database source, or from other sources, such as a Fortran file. Unformatted records include: • Optionally, a variable-length indicator bytes field • A variable-length data field Unformatted records have no end-of-record delimiter field. Table 2 lists the unformatted record field descriptions. Table 2: Unformatted Record Field Descriptions Input Record Field Description Indicator bytes Optional bytes to indicate null data Input data Raw data to be loaded into the Teradata FastLoad table on the Teradata Database Note: If the network-attached system configuration includes both UNIX and Windows operating system platforms, ensure that the Teradata FastLoad job scripts accommodate the different ways that each platform specifies new lines in ASCII text files. Windows platforms use a two-character sequence to signify a new line, carriage return + line feed. UNIX platforms use a single new-line character to perform the same function. When loading unformatted ASCII files from a network-attached system, always ensure that the DEFINE command properly identifies the format of the new-line function for the source platform: • One character for UNIX platforms • Two characters for Windows platforms Binary Data The binary data format is a 2-byte integer, n, followed by n bytes of data. Binary data format is similar to formatted data format, except that there is no end-of-record marker. Each record has: • 22 A two-byte data length field Teradata FastLoad Reference Chapter 1: Overview Input Data Formats • Optionally, a variable-length indicator bytes field • A variable-length input data field Table 3 lists the unformatted binary record field descriptions. Table 3: Binary Record Field Descriptions Input Record Field Description Data length A two-byte field indicating the total length of the record, in bytes, excluding the first two bytes (this field) The data length must be specified as an explicit value, not an ASCII value. For a data length of one byte, for example, the specification must be hexadecimal 01. It cannot be hexadecimal 31, or the ASCII equivalent of the number 1. Indicator bytes Optional bytes to indicate null data Input data The actual input data for rows in the Teradata FastLoad table. The input data stream always starts at either: • The third byte of the record, if there are no optional indicator bytes or • Immediately after the last optional indicator byte The length of the input data field, less any optional indicator bytes, is as specified in the first two bytes. Text Data TEXT specifies that each record consists of an arbitrary number of characters in the client session character set, followed by an end-of-record marker, which is: • On UNIX platforms, the newline character (identified in Unicode® as LINE FEED U+000A) • On Windows platforms, the two-character sequence carriage return followed by line feed (identified in Unicode as CARRIAGE RETURN U+000D and LINE FEED U+000A, respectively) For client session character sets other than UTF16, the end-of-record marker byte sequence is: • On UNIX platforms, X'0A' • On Windows platforms, X'0D0A' For the UTF16 client session character set (in which each character is encoded in two bytes), the end-of-record marker byte sequence is: • On big endian UNIX platforms, X'000A' • On little endian UNIX platforms, X'0A00' • On Windows platforms, X'0D000A00' Note: TEXT format should only be specified for character data. Do not specify TEXT format for binary data, such as, INTEGER, BYTEINT, PERIOD, and other binary data. Depending on the actual byte values of the binary data, unexpected results may occur. Teradata FastLoad Reference 23 Chapter 1: Overview Teradata FastLoad Commands Variable-length Text Variable-length text on a network-attached system is ASCII text data consisting of variable-length fields and records, with each field separated by a delimiter character. When loading variable-length text records from a network-attached system, use the SET RECORD command to specify the delimiter character. Unsupported Data Sets and Tapes Teradata FastLoad does not support the following types of data sets and tapes on mainframe-attached systems: • Concatenated data sets • Nonlabel (NL) and bypass label (BPL) tapes Teradata FastLoad Commands Teradata FastLoad accepts both Teradata FastLoad commands and a subset of Teradata SQL statements. The Teradata FastLoad commands perform session control and data-handling activities. The Teradata SQL statements define and manipulate the data stored in the Teradata Database. This section provides a summary of the Teradata FastLoad commands and Teradata SQL statements. Teradata FastLoad Command Summary The Teradata FastLoad commands perform two types of activities: • Session control – Session control commands begin and end Teradata FastLoad sessions and provide online information about a particular Teradata FastLoad operation. and • Data handling – Data handling commands establish and define a Teradata FastLoad operation. Table 4 and Table 5 summarize the Teradata FastLoad commands that perform these activities. For a detailed description of each Teradata FastLoad command, see Chapter 3: “Teradata FastLoad Commands.” Table 4: Teradata FastLoad Commands for Session Control Command Name Function HELP Lists Teradata FastLoad commands and options HELP TABLE Creates a list of field names which can be used with the INSERT statement LOGOFF Ends Teradata FastLoad sessions and terminates Teradata FastLoad QUIT 24 Teradata FastLoad Reference Chapter 1: Overview Teradata FastLoad Commands Table 4: Teradata FastLoad Commands for Session Control (continued) Command Name Function LOGON Begins one or more Teradata FastLoad sessions NOTIFY Specifies a user exit or action to be performed when certain significant events occur OS Enters client operating system commands SESSIONS Specifies the number of Teradata FastLoad sessions logged on with a LOGON command and, optionally, the minimum number of sessions required to run the job. If the database system (DBS) supports Teradata Active System Management (TASM), the number of sessions to run the jobs is determined by the DBS setup rules. Therefore, the SESSIONS command has no effect. For more information, see Database Administration (B035-1093). SHOW Shows the current field/file definitions established by DEFINE commands SHOW VERSIONS Shows the current level of all Teradata FastLoad software modules SLEEP Specifies the number of minutes that Teradata FastLoad pauses before retrying a logon operation TENACITY Specifies the number of hours that Teradata FastLoad continues trying to log on when the maximum number of load jobs is already running on the Teradata Database Table 5: Teradata FastLoad Commands for Data Handling Teradata FastLoad Command Function AXSMOD Specifies the name and initialization string for a shared object file that loads data from a file on network-attached client systems BEGIN LOADING Identifies the tables used in the Teradata FastLoad operation and, optionally, specifies when checkpoints are taken or if the user supplies indicator data CLEAR Cancels the current DEFINE command specifications DATEFORM Specifies the form of the DATE data type specifications for the Teradata FastLoad job DEFINE Describes each field of an input data source record and specifies the name of the input data source or INMOD routine END LOADING Informs the Teradata Database that all input data has been sent ERRLIMIT Limits the number of errors detected during the loading phase of a Teradata FastLoad job Processing stops when the limit is reached. RECORD Teradata FastLoad Reference Specifies the number of a record in an input data source at which Teradata FastLoad begins to read data and/or the number of the last record to be read 25 Chapter 1: Overview Teradata FastLoad Commands Table 5: Teradata FastLoad Commands for Data Handling (continued) Teradata FastLoad Command Function RUN Invokes the specified external source as the current source of commands and statements SET RECORD Specifies that the input data records are either: • • • • Formatted Unformatted Binary Text • Variable-length text Note: The SET RECORD command applies only to network-attached systems. SET SESSION CHARSET Specifies which character set is in effect during a specific Teradata FastLoad invocation Note: The SET SESSION CHARSET command applies only to network-attached systems. Note: The SET RETRY command is obsolete and ignored by Teradata FastLoad. For a description of the tenacity function, see “Invoking Teradata FastLoad” on page 33 and “TENACITY” on page 170. Teradata SQL Statements Teradata SQL statements define and manipulate the data stored in the Teradata Database. Table 6 summarizes the Teradata SQL statements supported by Teradata FastLoad Teradata FastLoad supports only the Teradata SQL statements listed in Table 6. To use other Teradata SQL statements, Teradata FastLoad must first be exited, and the statements entered from another application, such as Basic Teradata Query (BTEQ). Table 6: Teradata SQL Statements Supported in Teradata FastLoad Teradata SQL Statement Function CREATE TABLE Defines the columns, index, and other qualities of a table DATABASE Changes the default database DELETE Deletes rows from a table Note: FastLoad also supports temporal syntaxes like CURRENT VALIDTIME, SEQUENCED VALIDTIME, VALIDTIME, NONSEQUENCED VALIDTIME and NONTEMPORAL clauses prefixed in DELETE/DEL statement. DROP TABLE 26 Removes a table and all of its rows from a database Teradata FastLoad Reference Chapter 1: Overview Teradata FastLoad Example Table 6: Teradata SQL Statements Supported in Teradata FastLoad (continued) Teradata SQL Statement Function INSERT Inserts rows into a table SET QUERY_BAND...FOR SESSION Allows a set of name-value pairs to be defined by the user and/ or middle tier application so they can be customized to each application’s unique needs at the session level. Note: Although Teradata Database accepts the “SET QUERY_BAND ... FOR TRANSACTION;” , Teradata FastLoad rejects it, displays an error message and terminates. For syntax and a complete description of each Teradata SQL statement, see SQL Data Definition Language (B035-1144) and SQL Data Manipulation Language (B035-1146). The Teradata FastLoad version of the INSERT statement includes a special “wildcard” table name specification that is not supported by the Teradata Database. For a complete description of the Teradata FastLoad version of the INSERT statement, see “INSERT” on page 125. Teradata FastLoad Example This section provides an example of a small Teradata FastLoad job which can be quickly set up and run. The example shows how to: • Create a data file that will be used as the input source for a Teradata FastLoad job • Use a Teradata FastLoad job script to load data into a newly created table • Select data from the table to verify the load task Note: This example is for UNIX or Windows operating systems on a network-attached client system. Refer to the following appendixes for additional UNIX and Windows examples, and for z/OS examples on mainframe-attached systems: • Appendix B: “Multifile Teradata FastLoad Job Script Examples” • Appendix C: “INMOD and Notify Exit Routine Examples” • Appendix D: “Compile, Link, and Execute INMOD and Notify Exit Routines” Dropping the Employee Table The table in this example is named Employee. Use the Teradata SQL DROP TABLE command to delete any existing version of the Employee table from the database: bteq .logon tdpid/username,password DROP TABLE employee; .quit Creating the Source Data File Create and save a six-record text file named insert.input: Teradata FastLoad Reference 27 Chapter 1: Overview Teradata FastLoad Example |10021 |10001 |10002 |10028 |10029 |10023 |Brown, Jo |Jones, Bill |Smith, Jim |Lee, Sandra |Berg, Andy |Ayer, John |200|2312|Development |100|5376|President |100|4912|Sales |200|5844|Support |200|2312|Test |300|4432|Accounting |63000.00 |83000.00 |73000.00 |77000.00 |67000.00 |52000.00 |20|Jan |15|Jan |10|Jan | 4|Jan |10|Jan | 8|Jan 01 01 01 01 01 01 1955|F| 1960|M| 1970|M| 1971|F| 1967|M| 1965|M| |M|16| |M|14| |M|13| |M|18| |M|15| |M|13| 0| 0| 1| 0| 0| 0| Use this file as the input source for a Teradata FastLoad job. Note: The file example uses field delimiter characters ( | ) to help visualize each field. It is not required to use them in the file. Writing the Teradata FastLoad Job Script Create and save a Teradata FastLoad job script file named flinsert.fastload that loads the six-record insert.data file into the Employee table: sessions 2; errlimit 25; logon tdpid/username,password; CREATE TABLE employee ( EmpNo SMALLINT FORMAT ‘9(5)’ BETWEEN 10001 AND 32001 NOT NULL, Name VARCHAR(12), DeptNo SMALLINT FORMAT ‘999’ BETWEEN 100 AND 900 , PhoneNo SMALLINT FORMAT ‘9999’ BETWEEN 1000 AND 9999, JobTitle VARCHAR(12), Salary DECIMAL(8,2) FORMAT ‘ZZZ,ZZ9.99’ BETWEEN 1.00 AND 999000.00 , YrsExp BYTEINT FORMAT ‘Z9’ BETWEEN -99 AND 99 , DOB DATE FORMAT ‘MMMbDDbYYYY’, Sex CHAR(1) UPPERCASE, Race CHAR(1) UPPERCASE, MStat CHAR(1) UPPERCASE, EdLev BYTEINT FORMAT ‘Z9’ BETWEEN 0 AND 22, HCap BYTEINT FORMAT ‘Z9’ BETWEEN -99 AND 99 ) UNIQUE PRIMARY INDEX( EmpNo ) ; set record unformatted; define delim0(char(1)), EmpNo(char(9)), delim1(char(1)), Name(char(12)), delim2(char(1)), DeptNo(char(3)), delim3(char(1)), PhoneNo(char(4)), delim4(char(1)), JobTitle(char(12)), delim5(char(1)), Salary(char(9)), delim6(char(1)), YrsExp(char(2)), delim7(char(1)), DOB(char(11)), delim8(char(1)), Sex(char(1)), delim9(char(1)), Race(char(1)), delim10(char(1)), MStat(char(1)), delim11(char(1)), EdLev(char(2)), delim12(char(1)), HCap(char(2)), delim13(char(1)), newlinechar(char(1)) file=insert.input; show; begin loading employee errorfiles error_1, error_2; insert into employee ( :EmpNo, :Name, 28 Teradata FastLoad Reference Chapter 1: Overview Teradata FastLoad Example :DeptNo, :PhoneNo, :JobTitle, :Salary, :YrsExp, :DOB, :Sex, :Race, :MStat, :EdLev, :HCap ); end loading; logoff; Comments 1 For syntax and descriptions of the following commands, see Chapter 3: “Teradata FastLoad Commands”: • • • • • 2 SESSIONS ERRLIMIT LOGON SET RECORD DEFINE • • • • • SHOW BEGIN LOADING INSERT END LOADING LOGOFF The CREATE TABLE statement creates a new table on the Teradata Database: • Named employee • With 13 columns: • • • • • • • • EmpNo Name DeptNo PhoneNo JobTitle Salary YrsExp • • • • • • DOB Sex Race MStat EdLev HCap Indexed by EmpNo (UNIQUE PRIMARY) For syntax and a complete description of the Teradata SQL CREATE TABLE statement, see SQL Data Definition Language (B035-1144) and SQL Data Manipulation Language (B035-1146). 3 The DEFINE command specifies each field of the data records that will be sent to the Teradata Database. (The insert.input data file was created in the “Creating the Source Data File” subsection.) Each field definition provides the name and data type description for each field in the input data records. In making the field declarations, note that the one character delimiter fields are optional. They do not need to be used in the example. Teradata FastLoad Reference 29 Chapter 1: Overview Teradata FastLoad Example 4 The BEGIN LOADING command specifies the error files and starts the Teradata FastLoad job, using the insert.input file and the Employee table. Running the Teradata FastLoad Job Use the following command to invoke Teradata FastLoad and run the flinsert.fastload job script: fastload < flinsert.fastload Note that the redirection mechanism is required for Teradata FastLoad job scripts on network-attached client systems. Verifying the Import Task Use the following BTEQ commands to verify the Teradata FastLoad job by selecting newly loaded data from the Employee table: bteq .logon tdpid/username,password .width 120 select * from employee where salary > 65000.00; .quit The response indicates that the Employee table was successfully loaded with the data from the insert.input file: *** Query completed. 4 rows found. 13 columns returned. *** Total elapsed time was 6 seconds. EmpNo ----10028 10001 10002 10029 Name DeptNo PhoneNo ------------ ------ ------Lee, Sandra 200 5844 Jones, Bill 100 5376 Smith, Jim 100 4912 Berg, Andy 200 2312 JobTitle --------Support Preside Sales Test Salary YrsExp --------- -----77,000.00 4 83,000.00 15 73,000.00 10 67,000.00 10 DOB Sex Race MStat ----------- --- ---- ----JAN 01 1971 F M JAN 01 1960 M M JAN 01 1970 M M JAN 01 1967 M M EdLev ----18 14 13 15 HCap ---0 0 1 0 Other Alternatives The Teradata FastLoad example is a simple, straightforward exercise can be used to quickly create a job script and run Teradata FastLoad. The following information explains some of the more significant functional alternatives that Teradata FastLoad provides, and provides references to where they are described in greater detail. Mainframe-Attached Client Systems The Teradata FastLoad example assumes UNIX or Windows OS is the operating system on a network-attached client system. To run the example using z/OS on a mainframe-attached client system, use the standard z/OS JCL control statements (DD) to allocate and create the Teradata FastLoad data sets or files before invoking the utility. For more information about invoking Teradata FastLoad on mainframe-attached client systems, see “Invoking Teradata FastLoad” on page 33. 30 Teradata FastLoad Reference Chapter 1: Overview Teradata FastLoad Example MultiLoad Utility The Teradata FastLoad example shows a very simple Teradata FastLoad job, loading a very small amount of data into an empty table. MultiLoad could have been used to perform the same task, but the job would have run much more slowly. Teradata FastLoad works only on empty tables. MultiLoad can be used to: • Insert additional data rows into existing tables • Update individual rows of existing tables • Delete individual rows from existing tables • Load data into multiple tables • Load data into MULTISET tables • Load data into tables with nonunique secondary indexes Input File Formats The format of the input data source for the Teradata FastLoad example (insert.input) is UNFORMATTED, as specified by the SET RECORD command. Teradata FastLoad also supports input data source files with the following formats: • FORMATTED • BINARY • TEXT • VARTEXT For descriptions of all the supported input file formats, see “SET RECORD” on page 154. INMOD Routines In the Teradata FastLoad example, the utility reads the input data records directly from the specified source file (insert.input). An alternative would be to write an INMOD routine that Teradata FastLoad could call to obtain input records. The INMOD routine, for example, could: • Read and preprocess records from a file • Generate data records • Read data from other database systems • Validate data records • Convert data record fields In this case, use the optional INMOD name specification of the DEFINE command to identify the name of the INMOD routine. For more information about using INMOD routines, see “INMOD and Notify Exit Routines” on page 64 and the INMOD name option in “DEFINE” on page 97. Teradata FastLoad Reference 31 Chapter 1: Overview Teradata FastLoad Example 32 Teradata FastLoad Reference CHAPTER 2 Using Teradata FastLoad This chapter provides detailed information about using the Teradata FastLoad utility. Topics include: • Invoking Teradata FastLoad • Terminating Teradata FastLoad • Restart a Paused Teradata FastLoad Job • Programming Considerations • INMOD and Notify Exit Routines • Teradata FastLoad Job Scripts • Run Multifile Teradata FastLoad Jobs • Handling Teradata FastLoad Errors • Handling RDBMS Retryable and Restartable Error Codes • FastLoad TMSM Integration • FastLoad Statistics Invoking Teradata FastLoad This section describes invocation parameters and other factors related to starting and terminating Teradata FastLoad. File Requirements In addition to the input data source, Teradata FastLoad accesses four different data sets/files or input/output devices. Table 7 lists the FastLoad data sets/files and input/output devices. Table 7: FastLoad Data Sets/Files and Input/Output Devices Data Set/File or Device Provides standard input Teradata FastLoad commands and Teradata SQL statements that make up a Teradata FastLoad job standard output Destination for Teradata FastLoad output responses and messages standard error Destination for Teradata FastLoad error messages configuration Optional specification of Teradata FastLoad utility default values Teradata FastLoad Reference 33 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad When running Teradata FastLoad in interactive mode, the terminal keyboard functions as the standard input device and the display screen is the standard output/error device. When running Teradata FastLoad in batch mode, a data set or file name must be specified for each of these functions. The method of doing this varies, depending on the configuration of your client system: • On network-attached client systems, use the standard redirection mechanism (recommended approach) to specify the Teradata FastLoad script files when invoking the utility. Teradata FastLoad script files can also be piped to Teradata FastLoad when invoking the utility. • On mainframe-attached client systems, use standard z/OS JCL control commands to allocate and create the Teradata FastLoad data sets or files before invoking the utility. Interactive Mode To invoke Teradata FastLoad in interactive mode, enter fastload at the system command prompt: fastload Teradata FastLoad displays the following message to begin an interactive session: =================================================================== = = = FASTLOAD UTILITY VERSION 14.10.00.00 = = PLATFORM WIN32 = = = =================================================================== =================================================================== = = = Copyright 1984-2012, Teradata Corporation. = = ALL RIGHTS RESERVED. = = = =================================================================== Batch Mode This section covers invoking Teradata FastLoad in batch mode on network-attached and client-attached systems. Batch Mode on Network-Attached Client Systems Refer to the runtime parameter descriptions in Table 9 on page 39 and use the following syntax to invoke Teradata FastLoad in batch mode on network-attached client systems: 34 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad fastload -b kilobytes < infileName > outfileName -c characterSetName -d -e fileSame -M maxSessions -N minSessions -r outputRate -s minutes -t hours -v -y -i scriptEncoding -u outputEncoding -V 2411F007 Batch Mode on Mainframe-Attached z/OS Client Systems Refer to the runtime parameter descriptions in Table 8 on page 36, and use the following syntax to invoke Teradata FastLoad in batch mode on mainframe-attached z/OS client systems: //FASTLOAD EXEC PGM=FASTLOAD , ,PARM = ' BUFSIZE= kilobytes ' CHARSET= character-set-name DEBUG ERRLOG= filename MAXSESS =max-sessions MINSESS =min-sessions RATE =outputrate SLEEP=minutes TENACITY=hours VERBOSE RVERSION 2411D008 Mainframe-Attached Runtime Parameters Table 8 describes the runtime parameters used by Teradata FastLoad. Using these Teradata FastLoad runtime parameters will override configuration parameter settings. Teradata FastLoad Reference 35 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 8: Runtime Parameters (Mainframe-Attached Systems) Parameter Description BUFSIZE= kilobytes Output buffer size specification, where kilobytes is the size of the output buffer, in kilobytes, that will be used for Teradata FastLoad messages to the Teradata Database The output buffer size and the size of the rows in the Teradata FastLoad table determine the maximum number of rows that can be included in each message to the Teradata Database. A larger buffer size reduces processing overhead by including more data in each message. The default buffer size is also the maximum size allowed, 63 KB. If a value greater than 63 KB is specified, Teradata FastLoad: • Responds with a warning message • Resets the buffer size back to the default value • Continues with the Teradata FastLoad job 36 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 8: Runtime Parameters (Mainframe-Attached Systems) (continued) Parameter Description CHARSET= character-set-name A character set specification remains in effect for the entire Teradata FastLoad job, even if the Teradata Database server resets, causing the Teradata FastLoad job to be restarted. Caution: The character set specification does not remain in effect if the client system fails, or if the Teradata FastLoad job is cancelled. In these cases, when the job is resubmitted, he same character set specification that was used on the initial job must be used. If a different character set specification is used when such a job is resubmitted, the data loaded by the restarted job will not appear the same as the data loaded by the initial job. When a particular character set is specified, the identifiers and data in that character set can be specified, thus affecting how the data is loaded into the Teradata FastLoad table. If a character set specification is not entered, the default is whatever character set that is specified for the Teradata Database whenever Teradata FastLoad was invoked. The order in which the character set is determined for a Teradata FastLoad job is as follows: • User-specified by either a runtime parameter on mainframe-attached systems or a SET SESSION CHARSET command on network-attached systems • When using UTF-8 client character set on the mainframe, the client character set name needs to be specified by the runtime parameter (for example, CHARSET=UTF-8). UTF8 is also a valid value. • System Parameter Block (SPB) specified by either the HSHSPB on mainframe-attached systems or the clispb.dat file on network-attached systems • Teradata Database default, determined by a query from Teradata FastLoad Note: On IBM z/OS, the job script must be in Teradata EBCDIC when using UTF-8 client character set. Teradata FastLoad translates commands in the job script from Teradata EBCDIC to UTF-8 during the load. Be sure to examine the definition in International Character Set Support (B035-1132) to determine the code points of any special characters required. Different versions of EBCDIC do not always agree as to the placement of these characters. The mappings between Teradata EBCDIC and Unicode can be referred in International Character Set Support (B035-1132). DEBUG Teradata FastLoad Reference Used to display FastLoad execution debugging information to the standard output. 37 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 8: Runtime Parameters (Mainframe-Attached Systems) (continued) Parameter Description ERRORLOG= filename Alternate file specification for Teradata FastLoad error messages Specifying an alternate file name produces a duplicate record of all Teradata FastLoad error messages. The alternate file specification is limited to eight characters and, on z/OS, it must be to a DD name defined in the JCL. There is no default filename specification. MAXSESS = max-sessions Maximum number of Teradata FastLoad sessions logged on to the Teradata Database Maximum specification must be greater than zero and no more than the total number of AMPs on the system. Default is one session for each AMP. MINSESS = min-sessions Minimum number of Teradata FastLoad sessions required to run the job. Minimum specification must be greater than zero and it must be less than the value given for MAXSESS. Default is 1. RATE=outputrate The rate for displaying progress messages to standard output for rows successfully loaded to the Teradata Database. If not specified, FastLoad defaults to writing a message every 100000 rows. Standard output for the default looks similar to the following: **** 22:00:06 Starting row 400000 **** 22:00:11 Starting row 500000 **** 22:00:15 starting row 600000 outputrate must be specified as an integer between 1 and 4294967295. Integers outside this range or float values are invalid. If an invalid value is specified, FastLoad displays a message stating that an invalid outputrate value has been specified and the default value is used to process the Teradata FastLoad job. RVERSION Display version number and stop. This option must be run alone. Running the -V option with other run time options will produce invalid results. SLEEP= minutes Specification of the sleep option in which minutes is the number of minutes that Teradata FastLoad pauses before retrying the logon operation. For more information, see “SLEEP” on page 167. The Teradata FastLoad default value is 6 minutes. 38 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 8: Runtime Parameters (Mainframe-Attached Systems) (continued) Parameter Description TENACITY= hours Specification of the tenacity option in which hours is the number of hours that Teradata FastLoad continues trying to log on when the maximum number of load jobs are already running on the Teradata Database. For more information, see “TENACITY” on page 170. The Teradata FastLoad default is no tenacity. Either a Teradata FastLoad configuration file entry, the runtime parameter, or a TENACITY command in your Teradata FastLoad job script must be used to enable the tenacity feature for the Teradata FastLoad logon operation. VERBOSE Allows every request sent to the Teradata Database to be printed. Network-Attached Runtime Parameters Table 9 describes the runtime parameters used by Teradata FastLoad. Using these Teradata FastLoad runtime parameters will override configuration parameter settings. Table 9: Runtime Parameters (Network-Attached Systems) Parameter Description -b kilobytes Output buffer size specification, where kilobytes is the size of the output buffer, in kilobytes, that will be used for Teradata FastLoad messages to the Teradata Database. The output buffer size and the size of the rows in the Teradata FastLoad table determine the maximum number of rows that can be included in each message to the Teradata Database. A larger buffer size reduces processing overhead by including more data in each message. The default buffer size is also the maximum size allowed, 63 KB. If a value greater than 63 KB is specified, Teradata FastLoad: • Responds with a warning message • Resets the buffer size back to the default value • Continues with the Teradata FastLoad job Teradata FastLoad Reference 39 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description -c character-set-name Character set specification for the Teradata FastLoad job. A character set name, as described in “Character Set Specification” on page 55 can be specified. Character set specification remains in effect for the entire Teradata FastLoad job, even if the Teradata Database server resets, causing the Teradata FastLoad job to be restarted. Caution: The character set specification does not remain in effect if the client system fails, or if the Teradata FastLoad job is cancelled. In these cases, when a job is resubmitted, it must use the same character set specification that was used on the initial job. If a different character set specification is used when the job is resubmitted, the data loaded by the restarted job will not appear the same as the data loaded by the initial job. When a particular character set is specified, the identifiers and data in that character set can be specified, thus affecting how the data is loaded into the Teradata FastLoad table. If a character set specification is not entered, the default is whatever character set that is specified for the Teradata Database whenever Teradata FastLoad is invoked. Note: The order in which the character set is determined for a Teradata FastLoad job is as follows: 1 User-specified by either a runtime parameter on mainframe-attached systems or a SET SESSION CHARSET command on network-attached systems 2 System Parameter Block (SPB) specified by either the HSHSPB on mainframe-attached systems or the clispb.dat file on network-attached systems 3 Teradata Database default, determined by a query from Teradata FastLoad Note: When using UTF-16client character set on the network, the client character set name needs to be specified by the runtime parameter (for example, -c UTF-16). UTF16 is also a valid value. -d 40 Used to display FastLoad execution debugging information to the standard output Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description -i scriptencoding Specifies the encoding form of the job script. The parameter is introduced for use with the UTF-16 client character set, so it is only valid when UTF-16 client character set is used. If the client character set being used is not UTF-16 and the parameter is specified, Teradata FastLoad reports an error and terminates. Valid encoding options are: • • • • UTF-8 or UTF8 UTF-16BE, or UTF16-BE, or UTF16BE UTF-16LE, or UTF16-LE, or UTF16LE UTF-16 or UTF16 • UTF-8 indicates the job script is in UTF-8 character set. • UTF-16 indicates the job script is in UTF-16 character set without specifying the endianness. • UTF-16BE indicates the job script is in big endian UTF-16 character set. • UTF-16LE indicates the job script is in little endian UTF-16 character set. Or, if UTF-16LE is specified but the UTF-16 Byte Order Mark (BOM) in the script file indicates the script is in big endian, Teradata FastLoad reports an error and terminates. The UTF-16 or UTF-8 BOM can be present or absent in the script file. Specify the input script format with -i runtime parameter (mandatory) and specify session character set with -c runtime parameter (mandatory) to ensure that the BOM in the script file can be processed correctly. When UTF-16 is specified and the UTF-16 BOM is present in the script file, Teradata FastLoad interprets the script according to the endianness indicated by the UTF-16 BOM. When the UTF-16 BOM is not present, Teradata FastLoad interprets the script according to the endianness indicated by the option value. If the endianness is not indicated by the option value (for example, UTF-16 is specified instead of UTF16-BE or UTF-16LE), Teradata FastLoad interprets the job script in UTF-16 according to the endianness of the client system where the Teradata FastLoad job invoked. The specified encoding character set applies to all script files included by the .RUN FILE commands. Note: When this runtime parameter is not specified and UTF-16 client character is used, Teradata FastLoad interprets the job script in UTF-16. When UTF-8 is specified, Teradata FastLoad interprets the job script in UTF-8 and converts SQL and DML statements in the script from UTF-8 to UTF16 before sending the SQL and DML statements to Teradata Database. Teradata FastLoad Reference 41 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description Note: If the script is encoded in UTF-8 and the session character set specified is UTF-8, then -i UTF-8 must be specified, meaning the job must be run as “fastload -c UTF8 -i UTF8.” -u outputencoding Specifies the encoding form of the job output. The parameter is introduced for being used for UTF-16 client character set so it is only valid when UTF-16 client character set is used. If the client character set being used is not UTF-16 and the parameter is specified, Teradata FastLoad reports an error and terminates. Valid encoding options are: • • • • UTF-8 or UTF8 UTF-16BE, or UTF16-BE, or UTF16BE UTF-16LE, or UTF16-LE, or UTF16LE UTF-16 or UTF16 • UTF-16BE instructs Teradata FastLoad to print the job output in the big endian UTF16 character set. • UTF-16LE instructs Teradata FastLoad to print the job output in the little endian UTF-16 character set. • On big endian client systems, UTF-16 instructs Teradata FastLoad to print the job output in big endian UTF-16 character set. • On the little endian client systems, UTF-16 instructs Teradata FastLoad to print the job out in little endian UTF-16 character set. When the parameter is being used, it should be placed in front of the other runtime parameters to ensure the whole job output will be printed in the desired encoding form. If not placed ahead of the other runtime parameters when invoking the job, a warning message will be printed. When the parameter is not specified and the client character set being used is UTF-16, the job output will be printed as UTF-16. Note: Do not use Notepad to view Unicode characters and MBCS. Word provides the option to select the encoding when opening the file, and always displays the MBCSs correctly with the proper encoding. Other Hexidecimal editors can also be used to view Unicode characters and MBCS properly. 42 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description -e filename Alternate file specification for Teradata FastLoad error messages. Specifying an alternate file name produces a duplicate record of all Teradata FastLoad error messages. On z/OS mainframe-attached systems, the alternate file specification is limited to eight characters, and it must be to a DD name defined in the JCL. There is no default filename specification. When valid -u parameter is specified the contents of errorfile will be in the same output script encoding. For example, if UTF16 is specified for -u parameter, the contents are in Default Platform Endianness. When Little Endian is specified on Windows with Little Endian BOM in the beginning, or Big Endian is specified on a SPARC machine running Oracle Solaris with Big Endian BOM in the beginning. If UTF16-BE is specified for -u parameter, the contents are in Big Endian. < infilename Name of the standard input file that contains the Teradata FastLoad commands and Teradata SQL statements on network-attached client systems. The infilename specification redirects the standard input (stdin). If an infilename specification is not entered, the default is stdin. Caution: On a UNIX OS, if your infilename contains parenthesis, commas, or equal signs, FastLoad will return an error of “File Not Found”. The infilename must be changed to remove these special characters. On Windows, these characters are legal, and therefore no changes are necessary. Note: On mainframe-attached client systems, the SYSIN control command must be used to specify the input file before the utility is invoked. Teradata FastLoad Reference 43 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description > outfilename Name of the standard output file for Teradata FastLoad messages on network-attached systems. The outfilename specification redirects the standard output (stdout). If an outfilename specification is not entered, the default is stdout. Caution: If an outfilename specification is not used to redirect stdout, do not use the same outfilename as an output or echo destination in the ROUTE MESSAGES command. Doing so produces incomplete results because of the conflicting write operations to the same file. Note: On mainframe-attached client systems, the SYSPRINT control command must be used to specify the output file before invoking the utility. Specification on mainframe-attached systems that the Teradata FastLoad job will use an INMOD routine written in the IBM C programming language. The Teradata FastLoad job will fail if this option is not specified when using a IBM C INMOD routine. -M max-sessions Maximum number of Teradata FastLoad sessions logged on to the Teradata Database. Maximum specification must be greater than zero and no more than the total number of AMPs on the system. Default is one session for each AMP. -N min-sessions Minimum number of Teradata FastLoad sessions required to run the job. Minimum specification must be greater than zero and it must be less than the value given for MAXSESS. Default is 1. -s minutes Specification of the sleep option in which minutes is the number of minutes that Teradata FastLoad pauses before retrying the logon operation. For more information, see “SLEEP” on page 167. The Teradata FastLoad default value is 6 minutes. 44 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad Table 9: Runtime Parameters (Network-Attached Systems) (continued) Parameter Description -t hours Specification of the tenacity option in which hours is the number of hours that Teradata FastLoad continues trying to log on when the maximum number of load jobs are already running on the Teradata Database. For more information, see “TENACITY” on page 170. The Teradata FastLoad default is no tenacity. Either a Teradata FastLoad configuration file entry, the runtime parameter, or a TENACITY command in the Teradata FastLoad job script must be used to enable the tenacity feature for the Teradata FastLoad logon operation. -v Allows printing of every request sent to the Teradata Database. -y Specification for the data encryption option. When specified at run time, all sessions will be encrypted. -V Display version number and stop. This option must be run alone. Running the -V option with other run time options will produce invalid results. -w FastLoad wraps long error message to cleanup the message. “-w” allows a user to specify the output message width. The output width can be set to any value from 62 to 256 bytes. If used, FastLoad wraps long error messages at the width specified, instead of the at 72 bytes if the option is not set for backward compatibility. If an out of range value is set, FastLoad issues a warning message and continues to use default constant width for output messages. -r outputrate The rate for displaying progress messages to standard output for rows successfully loaded to the Teradata Database. If not specified, FastLoad defaults to writing a message every 100000 rows. Standard output for the default looks similar to the following: **** 22:00:06 Starting row 400000 **** 22:00:11 Starting row 500000 **** 22:00:15 starting row 600000 outputrate must be specified as an integer between 1 and 4294967295. Integers outside t-s range or float values are invalid. If an invalid value is specified, FastLoad displays a message stating that an invalid outputrate value has been, specified and the default value is used to process the Teradata FastLoad job. Note: The first occurrence of the runtime parameter is in effect if the duplicates are specified and the remaining will be omitted with a warning message “FDL2430 WARNING: Run time parameters detected and omitted.” Teradata FastLoad Reference 45 Chapter 2: Using Teradata FastLoad Invoking Teradata FastLoad z/OS Example Following is an example z/OS program that invokes Teradata FastLoad: //FASTLOAD EXEC TDSFAST,INFILE=’<input dataset>’ //FAST.SYSIN DD DATA,DLM=$$ <FastLoad Control Statements> $$ where the ddname SYSIN is the input data set that contains the Teradata FastLoad job script. z/OS Procedure Following is a sample z/OS procedure that is provided on the release tape with Teradata FastLoad software. This procedure can be changed to meet specific site requirements and use it to invoke Teradata FastLoad on a mainframe-attached z/OS client system. //TDSFAST PROC FDLOPT=,PFX1=’DBC’,PFX2=’DBC’, //INFILE=’NULLFILE’ //***************************************** //* THIS PROCEDURE EXECUTES * //* THE FASTLOAD PROGRAM * //* * //* YOU CAN COPY THIS PROCEDURE * //* INTO YOUR SYSTEM PROCLIB * //* FOR ALL DBC USERS GENERAL USAGE. * //* * //* EXAMPLE EXECUTION: * //* //FAST EXEC TDSFAST * //* //FAST.SYSIN DD DATA,DLM=$$ * //* LOGON .....; * //* LOGOFF; * //* $$ * //* * //* DDNAMES USED: * * //* SYSPRINT - PRINTED OUTPUT. //* SYSIN - INPUT DATA SET * //* CONTAINING FASTLOAD * //* STATEMENTS. * //* INFILE - INPUT DATA SET * //* CONTAINING DATA. * //*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* //* TAILOR THE FOLLOWING FOR YOUR * //* SPECIFIC ENVIRONMENT: * //* SYMBOLIC USE * //* ======== === * //* PFX1 = HIGH LEVEL QUALIFIER * //* FOR APPLOAD LIBRARY. * //* FPX2 = HIGH LEVEL QUALIFIER * //* FOR IBM C RUNTIME LIBRRARY.* //* FDLOPT = FASTLOAD PARAMETER INPUT. * //* INFILE = DSNAME OF INPUT DATASET. * //* * //*---------------------------------------* //* * //* -DATE-- III DR/DCR- CHNG#--COMMENTS-- * //* * //* 23JAN96 MXM CREATED FOR C * 46 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Terminating Teradata FastLoad //* VERSION OF * //* FASTLOAD * //***************************************** //FASTLOAD EXEC PGM=FASTLOAD, // PARM=’&FDLOPT’, REGION=4096K //STEPLIB DD DSN=&PFX1..APPLOAD,DISP=SHR // DD DSN-&PFX2..TRLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSABEND DD SYSOUT=* //SYSTERM DD SYSOUT=* //INFILE DD DSN=&INFILE,DISP=SHR UNIX and Windows Examples Following are examples of three ways to invoke Teradata FastLoad on network-attached client systems: • fastload </home/fluser/tests/test1 >/home/fluser/tests/out1 This command specifies both an input file and an output file: • /home/fluser/tests/test1 is the input file that provides the Teradata FastLoad job script. • • /home/fluser/tests/out1 is the destination file for output data. fastload </home/fluser/tests/test1 This command specifies only an input file. In this case, the output is written to the standard output device, which is usually a terminal. • fastload This command specifies neither an input nor an output device. In this case, the terminal provides both the command input and the output data destination. Terminating Teradata FastLoad This section covers methods of termination and other topics related to terminating Teradata FastLoad. Definition There are two ways to terminate Teradata FastLoad, depending on the operational situation: • Normal Termination or • Abort Termination Either way ends all Teradata FastLoad sessions and logs off the Teradata Database. A normal termination, however, does so in an orderly, controlled fashion, and returns messages indicating the status of the Teradata FastLoad job, and whether the utility was paused or terminated. An abort termination does not. Teradata FastLoad Reference 47 Chapter 2: Using Teradata FastLoad Terminating Teradata FastLoad Normal Termination Use the LOGOFF or QUIT command in a Teradata FastLoad batch job script or interactive session to terminate Teradata FastLoad normally on both network-attached and mainframe-attached client systems: LOGOFF ; QUIT 2411A013 Teradata FastLoad logs off all sessions with the Teradata Database and returns a status message indicating: • The total processor time that was used. • The job start and stop date/time. • The highest return code that was encountered: • • 0 if the job completed normally • 4 if a warning condition occurred • 8 if a user error occurred • 12 if a fatal error occurred Whether the utility terminated or paused For more information about return codes and the conditions that terminate or pause Teradata FastLoad, see “LOGOFF” on page 131 or “QUIT” on page 145. Abort Termination The procedure for aborting a Teradata FastLoad job depends on whether the utility is running on a network-attached or mainframe-attached client system. To abort a Teradata FastLoad job running on a network-attached client system ✔ Press the Control + C key combination three times on the workstation keyboard. Note: Some sessions of the aborted Teradata FastLoad job may remain active until the gateway timeout period expires. Error “2738 %TVMID already has Teradata FastLoad running” may occur if the same Teradata FastLoad job is resubmitted within the timeout period. If this happens, wait until the gateway time-out has expired before resubmitting the same Teradata FastLoad job. To abort a Teradata FastLoad job running on a mainframe-attached client system ✔ Cancel the job from the client system console. 48 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Restart a Paused Teradata FastLoad Job Restart a Paused Teradata FastLoad Job This section describes restarting Teradata FastLoad jobs that have been paused or interrupted. Overview A paused Teradata FastLoad job is one that was halted, before completing, during the loading or end loading phase of the Teradata FastLoad operation. The paused condition can be intentional, or the result of a system failure or error condition. A Teradata FastLoad job can be paused intentionally by using the LOGOFF or QUIT command before the END LOADING command in the Teradata FastLoad job script, as when running a multifile Teradata FastLoad job. Starting from Teradata Tools and Utilities 14.00, there is a data signature included in checkpoint information. That data signature is validated during a restart and may result in new errors at the beginning of a restart. Unintentional conditions that can pause a Teradata FastLoad job include: • Client system or Teradata FastLoad failures • Unrecoverable error conditions • Database or table overfills • Teradata Database failures When a Teradata FastLoad job is in the paused state, the Teradata FastLoad target table and the two error tables on the Teradata Database are locked. Two error tables can be accessed by using a locking modifier, such as: After a Client System or Teradata FastLoad Failure When a client system or a Teradata FastLoad failure occurs while a Teradata FastLoad job is running, Teradata FastLoad: 1 Pauses the job. 2 Tries to log off all Teradata FastLoad sessions on the Teradata Database. locking error_table_name for access select errorcode, errorfieldname from error_table_name; To access the Teradata FastLoad target table, a Teradata FastLoad job consisting of a BEGIN LOADING and an END LOADING command must be submitted. This effectively restarts and ends the job. After executing the END LOADING command, the Teradata FastLoad target table can be accessed, but the original job cannot be restarted. The following subsections describe the various pause conditions, the factors affecting Teradata FastLoad restart operations, and the procedure for restarting a paused Teradata FastLoad job. Teradata FastLoad Reference 49 Chapter 2: Using Teradata FastLoad Restart a Paused Teradata FastLoad Job After a Database Overfill Condition When a Teradata FastLoad job tries to load more data into a table than the table or the database that it is in can hold, Teradata FastLoad pauses the job and returns the following message: RDBMS error 2644: No more room in database <uid> Increase database size and restart Teradata FastLoad In this case, after increasing the amount of space allocated to the database, the Teradata FastLoad job can be restarted at the point where the reported out-of-space condition occurred. After a Teradata Database Failure Restarting a Teradata FastLoad job that was paused because of a Teradata Database failure depends on the operational configuration of the Teradata Database when it returns to service: • If the configuration of the restarted Teradata Database is exactly the same as it was when Teradata FastLoad was invoked, then Teradata FastLoad restarts the job automatically. In this case, if the Teradata FastLoad job was paused in the end loading phase, the Teradata Database resumes processing at the same place it was stopped. If the Teradata FastLoad job was paused in the loading phase, the Teradata Database resumes processing: • at the last checkpoint if the BEGIN LOADING command specified the checkpoint option. • at the beginning if the BEGIN LOADING command did not specify the checkpoint option. Note: If the Teradata FastLoad job uses an INMOD routine, the routine must be able to handle restarts and checkpoints when restarted in the loading phase. • If the configuration of the restarted Teradata Database is different from the way it was when Teradata FastLoad is invoked, then Teradata FastLoad does not restart the job. In this case, to restart and continue with the paused Teradata FastLoad job, must reestablish the original configuration of the Teradata Database. If this is not possible, then the Teradata FastLoad job must: a Delete the Teradata FastLoad table and error tables. b Resubmit the Teradata FastLoad job, from the beginning, as a new job. After an Unrecoverable Error Condition When an unrecoverable error condition occurs while a Teradata FastLoad job is processing the BEGIN LOADING command, but before completing the end loading process, Teradata FastLoad: 50 1 Pauses the job. 2 Logs off all Teradata FastLoad sessions on the Teradata Database. Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Restart a Paused Teradata FastLoad Job 3 Writes the Teradata FastLoad PAUSED message to the standard output device, along with the error code associated with the error. Note: Teradata FastLoad terminates if an unrecoverable error condition occurs before it begins processing the BEGIN LOADING command, or after it has completed processing the END LOADING command. Note: FastLoad does not support the restart of a Name Pipe Access Module after the job aborted in a restart condition. Restart Procedures The procedure which is used and the Teradata FastLoad response to restarting a paused Teradata FastLoad job depends on the phase that the Teradata FastLoad job was in when it was paused. • A job that was paused during loading can be restarted, either from the beginning, or from the most recent checkpoint if the BEGIN LOADING command specified the checkpoint option. • Or, a job that was paused during end loading from wherever it was interrupted can be restarted. This is because the Teradata Database uses internal checkpointing during this phase. Note: Generally, nothing needs to be done because processing after the END LOADING command is executed on the Teradata Database does not depend on Teradata FastLoad. However, if restarting the job in the end loading phase is required, see the following procedure. To restart the job if the Teradata FastLoad job was paused during the loading phase 1 Remove the CREATE TABLE statement and any DROP TABLE and DELETE statements from the Teradata FastLoad job script to prevent the restarted job from dropping the partially loaded Teradata FastLoad table or deleting the entries in the two error tables. 2 Invoke Teradata FastLoad to start the job. The Teradata FastLoad utility: • Establishes new sessions using the LOGON command. • Reads the restart log to determine the restart point. • In response to the BEGIN LOADING command, indicates that the job is being restarted. If, for example, the job had a checkpoint specification of 100, and the failure occurred at row 1100, the Teradata FastLoad response would be: FastLoad RESTARTED The last checkpoint was taken at row: 1100 FastLoad will now restart at row: 1101 Note: If the Teradata FastLoad job was paused during the loading phase and uses an INMOD routine, the INMOD routine must be able to handle restarts and checkpoints when restarted in the loading phase. Following a restart, Teradata FastLoad passes a status code of 2 or 4 to an INMOD routine that is participating in a load operation. The routine must adjust the record Teradata FastLoad Reference 51 Chapter 2: Using Teradata FastLoad Programming Considerations to be read to the position of the last checkpoint and, upon subsequent calls, send records to Teradata FastLoad. To restart the job if the Teradata FastLoad job was paused during the end loading phase 1 Use the same LOGON command described in the preceding procedure. 2 Submit BEGIN LOADING and END LOADING commands, as in the following example: LOGON dbc/sjn,music ; BEGIN LOADING Fast_Table ERRORFILES Error_1, Error_2 ; END LOADING ; Note: If a Teradata FastLoad job script is used to assemble these commands, make sure to delete the CREATE TABLE and any DROP TABLE and DELETE statements before restarting the job. Programming Considerations This section describes the things to consider when designing and coding a Teradata FastLoad job script. Teradata FastLoad Configuration File A Teradata FastLoad configuration file can be created to set the initial default values for the following operating parameters when Teradata FastLoad is invoked: • TENACITY • SLEEP • BUFSIZE • CHARSET • INMODRETURN • MAXSESS • MINSESS • CONFIGERRORS • RETRYFIRSTCONNECT • RETRYOTHERCONNECT • RETRYCONNECTINTERVAL Note: RETRYFIRSTCONNECT, RETRYOTHERCONNECT and RETRYCONNECTINTERVAL can only be specified in the configuration file for networkattached platforms, there are no corresponding command line options. The values specified in the Teradata FastLoad configuration file override the internal utility default values for these parameters. 52 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations The configuration file parameters themselves can be overridden by the TENACITY, SLEEP, SESSION, and SET SESSION CHARSET commands in the Teradata FastLoad job script, and by the corresponding runtime parameters, as shown in Table 8 on page 36 and Table 9 on page 39. The order of preferences for the TENACITY, SLEEP and SESSION specifications, from the highest to lowest, is: 1 Runtime parameters 2 Teradata FastLoad script commands 3 Teradata FastLoad configuration file specifications 4 Teradata FastLoad default values Note: The utility default for TENACITY is no tenacity. Either a configuration file entry, the runtime parameter, or the TENACITY command in your Teradata FastLoad job script must be used to enable the tenacity feature for the Teradata FastLoad logon operation. The order of preferences for the CHARSET specification, from the highest to lowest, is: 1 Runtime parameters 2 Teradata FastLoad script commands 3 Teradata FastLoad configuration file specifications 4 Teradata FastLoad default values Configuration File Name and Location On network-attached systems, the Teradata FastLoad configuration file must be named: floadcfg.dat And it must be located in either: • The directory from which Teradata FastLoad was launched • The directory specified in the FLOADLIB environment variable On mainframe-attached systems, the DD statement for the Teradata FastLoad configuration file must be labeled FLOADCFG Configuration File Contents The Teradata FastLoad configuration file can have up to twelve entries, one for each parameter: TENACITY=hours SLEEP=minutes BUFSIZE=kilobytes CHARSET=character-set-name INMODRETURN=ON or YES MAXSESS=max-sessions MINSESS=min-sessions DATAENCRYPTION=ON or OFF CONFIGERRORS=IGNORE/TERMINATE RETRYFIRSTCONNECT=n RETRYOTHERCONNECT=n RETRYCONNECTINTERVAL=s Teradata FastLoad Reference 53 Chapter 2: Using Teradata FastLoad Programming Considerations where: • hours is the TENACITY specification of the number of hours that Teradata FastLoad continues trying to log on when the maximum number of load jobs are already running on the Teradata Database. For more information about this specification, see “TENACITY” on page 170. • minutes is the SLEEP specification of the number of minutes that Teradata FastLoad pauses between attempts to log on. For more information about this specification, see “SLEEP” on page 167. • kilobytes is the size of the output buffer, in kilobytes, to be used for Teradata FastLoad messages to the Teradata Database. For a description of the BUFSIZE specification, see Table 8 on page 36. • character-set-name is the character set specification for the job. For more information about this specification, see Table 8 on page 36 orTable 9 on page 39 and “SET SESSION CHARSET” on page 161. • ON or YES specifies that INMOD return codes are to be checked and returned. The informational message INMOD return codes will be checked displays. Nonzero return codes force Teradata FastLoad to terminate. • max-sessions is the MAXSESS specification for the maximum number of Teradata FastLoad sessions logged on to the Teradata Database. For more information about this specification, see “SESSIONS” on page 151. • min-sessions is the MINSESS specification for the minimum number of Teradata FastLoad sessions required to run the job. For more information about this specification, see “SESSIONS” on page 151. • RETRYFIRSTCONNECT, RETRYOTHERCONNECT and RETRYCONNECTINTERVAL are used to deal with CLI Error 207 return (network down when trying to connect). RETRYFIRSTCONNECT is used to specify number of retries for the 1st SQL main connection (the default is 0). RETRYOTHERCONNECT is used to specify the number of retries for other connections including auxiliary and fastload session connection (the default is 16). RETRYCONNECTINTERVAL is used to specify the interval between retries (the default is 60 second), the value specify is in term of seconds. • ON or OFF specifies whether data encryption will be enabled for the job. • IGNORE/TERMINATE configures the option for configuration file error handling. The configuration file can also have comment statements preceded by a number sign (#) character. Configuration File Processing Teradata FastLoad automatically checks for a configuration file each time the invocation command is entered. Upon locating a configuration file, the utility sets the defaults as specified, produces the appropriate output messages, and begins processing the Teradata FastLoad job. By default, any invalid configuration file entry or syntax error immediately aborts a job and produces return code 12. The first invalid parameter is reported; none of the other configuration parameters are checked. 54 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations • If CONFIGERRORS=IGNORE is specified, any subsequent configuration file problems will be reported, but they will not affect the return code. FastLoad will continue to process the next entry in the configuration file. • If CONFIGERRORS=TERMINATE is specified (the default), any subsequent invalid configuration file entry will abort the job. The CONFIGERRORS value can be changed more than once in a configuration file. Its value affects subsequent entries processing until the next CONFIGERRORS entry or until the end of the configuration file. If no configuration file exists, the utility begins processing the load job without an error indication. The configuration file is an optional feature; its absence is not considered an error condition. Note: The config file must be in the platform-appropriate single-byte character set, like ASCII or EBCDIC. Character Set Specification Table 10 lists character sets supported by Teradata FastLoad. Table 10: Character Sets Supported by Teradata FastLoad Character Set Name Description Configuration ASCII Latin Network-attached EBCDIC Latin Mainframe-attached HANGULEBCDIC933_1II Korean Mainframe-attached HANGULKSC5601_2R4 Korean Network-attached KANJIEUC_0U Japanese Network-attached KANJISJIS_0S Japanese Network-attached KATAKANAEBCDIC Japanese Mainframe-attached KANJIEBCDIC5026_0I Japanese Mainframe-attached KANJIEBCDIC5035_0I Japanese Mainframe-attached SCHEBCDIC935_2lJ Simplified Chinese Mainframe-attached SCHGB2312_1T0 Simplified Chinese Network-attached TCHEBCDIC937_3IB Traditional Chinese Mainframe-attached TCHBIG5_1R0 Traditional Chinese Network-attached UTF-8 Unicode character set • Mainframe-attached • Network-attached UTF-16 Teradata FastLoad Reference Unicode character set Network-attached 55 Chapter 2: Using Teradata FastLoad Programming Considerations Site-Defined Character Sets When the character sets defined are not appropriate for a site, custom character sets can be defined using the information in Table 11. For information on defining a custom character set, see SQL Data Definition Language (B035-1144). Table 11 lists the site-defined character sets. Table 11: Site-Defined Character Sets Character Set Name Description Configuration SDHANGULEBCDIC933_5II Korean Mainframe-Attached SDHANGULKSC5601_4R4 Korean Network-Attached SDKATAKANAEBCDIC_4IF Japanese Mainframe-Attached SDKANJIEBCDIC5026_4IG Japanese Mainframe-Attached SDKANJIEBCDIC5035_4IH Japanese Mainframe-Attached SDKANJIEUC_1U3 Japanese Network-Attached SDKANJISJIS_1S3 Japanese Network-Attached SDSCHEBCDIC935_6IJ Simplified Chinese Mainframe-Attached SDTCHEBCDIC937_7IB Traditional Chinese Mainframe-Attached SDSCHGB2312_2T0 Simplified Chinese Network-Attached SDTCHBIG5_3R0 Traditional Chinese Network-Attached Note: For information on defining a custom character set, see International Character Set Support (B035-1132). Rules for Chinese and Korean Character Set Use Follow these rules when using Chinese and Korean character sets on mainframe-attached and network-attached platforms. • Object Names Object names are limited to A-Z, a-z, 0-9, and special characters such as $ and _. • Maximum String Length The DBS requires two bytes to process each of the Chinese or Korean characters. This limits both request size and record size. For example, if a record consists of one string, the length of that string is limited to a maximum of 32,000 characters or 64,000 bytes. For more information about Chinese or Korean character set restrictions for the Teradata Database, refer to International Character Set Support (B035-1132). 56 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations AXSMOD When an AXSMOD is used, Teradata FastLoad will pass the session character set as an attribute to the AXSMOD for its possible use (most AXSMODs will not make any use of this information). The attribute value will be a variable length character string consisting of the character set name; the attribute name will be CHARSET_NAME. Unicode® Character Sets UTF-8 and UTF-16 are two of the standard ways of encoding Unicode character data. The UTF-8 client character set supports UTF-8 encoding. Currently Teradata Database supports UTF-8 characters that can consist of from one to three bytes. The UTF-16 client character set supports UTF-16 encoding. Currently, the Teradata Database supports the Unicode 5.1 standard, where each defined character requires exactly 16 bits. For restrictions imposed by Teradata Database on the use of UTF-8 or UTF-16 character set, see International Character Set Support (B035-1132). UTF-8 Character Sets Teradata FastLoad supports UTF-8 character set on network-attached platforms and IBM z/ OS. On IBM z/OS, the job script must be in Teradata EBCDIC when using UTF-8 client character set. Teradata FastLoad translates commands in the job script from Teradata EBCDIC to UTF-8 during the load. Be sure to examine the definition in International Character Set Support (B035-1132) to determine the code points of any special characters which might be required in the job script. Different versions of EBCDIC do not always agree as to the placement of these characters. Refer to the mappings between Teradata EBCDIC and Unicode in International Character Set Support (B035-1132). UTF-16 Character Sets Teradata FastLoad supports UTF-16 character set on network-attached platforms. In general, the command language and the job output should be the same as the client character set used by the job. However, for user’s convenience and because of the special property of Unicode, the command language and the job output are not required to be the same as the client character set when using UTF-16 character set. When using UTF-16 character set, the job script and the job output can either be in UTF-8 or UTF-16 character set. This is provided by specifying runtime parameters “-i” and “-u” when the job is invoked. ANSI/SQL DateTime Specifications The ANSI/SQL DATE, TIME, TIMESTAMP and INTERVAL DateTime data types in Teradata SQL CREATE TABLE statements, can be used. Specify them as column/field modifiers in INSERT statements. However, certain restrictions should be noted: • Teradata FastLoad Reference ANSI/SQL DateTime data types cannot be used when specifying the column/field names in a DEFINE command. 57 Chapter 2: Using Teradata FastLoad Programming Considerations • ANSI/SQL DateTime data types must be converted to fixed-length CHAR data types when specifying the column/field names in the DEFINE command. For a description of the fixed-length CHAR representations for each DATE, TIME, TIMESTAMP and INTERVAL data type specification, see Table 27 on page 96. Checkpoint Tradeoffs Though the checkpoint feature substantially enhances Teradata FastLoad restart operations, it also reduces the speed of the Teradata FastLoad job. Always consider the following factors when deciding how often to specify checkpoints: • Each checkpoint temporarily halts the multiple session data transfer feature of Teradata FastLoad, thereby decreasing the speed of the Teradata FastLoad job. • For each checkpoint, Teradata FastLoad waits for each session to complete sending its current request, which interrupts the data transfer operation. • The record size and the size of the Teradata Database influence how often checkpoints should be specified. On a smaller Teradata Database, specify checkpoints: • Every 50,000 records if each record is more then 4 KB • Every 100,000 records if each record is less than 4 KB On a larger Teradata Database, specify higher (less frequent) checkpoint values. Checkpoint Support with Access Modules When an AXSMOD is used, Teradata FastLoad will pass the checkpoint value as an attribute to the AXSMOD for its possible use. The attribute value will be a string containing the ASCII encoding of the decimal textual representation of the checkpoint value. A value of “0” indicates that Teradata FastLoad is not using checkpoints. A file opened by an access module instance for which the CHECKPOINT_INTERVAL attribute is “0” will not receive a File Get Position request. Comments Teradata FastLoad supports C program language comments. Comments supported for Teradata FastLoad have the following characteristics: • Begin with a forward slash asterisk (/*) character sequence and end with an asterisk forward slash (*/) sequence. These sequences are the beginning and ending delimiters; all intervening text is treated as a comment. • Will be sent to the Teradata Database • Are invalid within string or character literals. A /* within a quoted string is not treated as the beginning of a comment. Teradata FastLoad does not support nested comments. 58 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations Concurrent Load Utility Tasks The maximum number of concurrent Teradata FastLoad tasks that can run is variable; the limit can be controlled by a system administrator. MaxLoadTasks may be overridden if Teradata Active System Management (Teradata ASM) is active. For the most up-to-date information on concurrent task limits, see the description of the MaxLoadTask parameter of the DBSControl utility inUtilities (B035-1102). Additional information is available in the Teradata Dynamic Workload Manager User Guide (B035-2513). If a Teradata FastLoad job exceeds recommended limits, Teradata FastLoad returns a 2633 error message indicating that too many loads are running. In this case, the utility retries until: • It can execute the task. • It reaches the TENACITY hours time limit specified by the BEGIN EXPORT command or runtime parameter. Data Conversion Factors Be aware of the following when using Teradata FastLoad to perform data conversion: • When using the DEFINE command to change the data type specification of source data before inserting it into the Teradata FastLoad table on the Teradata Database, only one type conversion per column can be specified. Note: Teradata FastLoad cannot be used to define a column with an arithmetic expression. For example, Teradata FastLoad will not calculate a monthly salary column from yearly salary data. • When using Teradata FastLoad to load and convert data from decimal to character, define the length of the column into which the data will be inserted to be larger than the size of the source data in order to accommodate the sign byte. Otherwise, the Teradata Database generates an error message as follows: 3944: Data length is invalid for the data type. For details on data types and data conversions, see SQL Data Types and Literals (B035-1143). Valid Data Conversion Example The following valid example converts character data to integer data, assuming that column b is of type INTEGER: DEFINE a (char(10)) file= . . . ; INSERT INTO table1 (b) VALUES (:a(integer)) ; Valid Redundant Conversion Example The following valid example converts data in zoned decimal format to type decimal format, assuming that column d is of type DECIMAL (9,2): DEFINE b (char(9)) file= . . . ; INSERT INTO Table1 (d) VALUES (:b (decimal(9,2))); Note: Redundant conversions can be used, as a reminder that a data conversion is taking place. Teradata FastLoad Reference 59 Chapter 2: Using Teradata FastLoad Programming Considerations Invalid Data Conversion Example The following data conversion example is invalid because it requests two conversions, from character to decimal, and then from decimal to integer, assuming that column b is of type INTEGER: DEFINE a(char(10)) file= . . . ; INSERT INTO table1 (b) VALUES (:a (decimal(5,2))) ; If loading Decimal(5,0) data, the length of the destination character column must be defined as Char(6) instead of Char(5). The following data conversion example is invalid because the column size is not large enough to hold all of the data: DEFINE a(decimal(5,0)) file = .; INSERT INTO table (b) VALUES (: a(char(5)); Error Limits When specifying the ERRLIMIT value, consider the number and type of errors expected from the Teradata FastLoad job. • If the Teradata FastLoad job is expected to encounter no errors or very few errors, then specify an ERRLIMIT value that is low. • If the Teradata FastLoad job is expected to encounter many errors that are considered allowable, then specify an ERRLIMIT value that is high. Nonunique Index Sorts If a Teradata FastLoad table is defined with a nonunique primary index, the performance of the Teradata FastLoad job can be enhanced by not using the index value to sort the input data. Duplicate Rows Teradata FastLoad does not load duplicate rows, as in MULTISET tables. If Teradata FastLoad is used to load a target table defined as MULTISET, the utility will discard any duplicate rows. If duplicate rows must be loaded, consider using MultiLoad. Range Constraints Range constraints are data description phrases that are entered into the Teradata SQL CREATE TABLE statement that limit the range of acceptable values for a column. Since the range constraint checks occur while Teradata FastLoad inserts data into the Teradata FastLoad table, the number of range constraints in the Teradata FastLoad job script has a direct impact on the performance of Teradata FastLoad. Range Constraint Types Table 12 list the two types of range constraints, explicit and implicit. 60 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations Table 12: Range Constraint Types Constraint Type Example Explicit The Salary column range of between 1 and 99000.00, as shown in the following CREATE TABLE example. Implicit The DeptNo column range of ZZ9, as shown in the following CREATE TABLE example. Range Constraint Examples The following Teradata SQL CREATE TABLE statement shows the two types of range constraint phrases for the Salary and DeptNo columns: CREATE TABLE Employee (EmpNo INTEGER FORMAT ‘ZZZZ9’, Name VARCHAR (12) CASESPECIFIC TITLE ‘Employee//Name’, DeptNo INTEGER FORMAT ‘zz9’ TITLE ‘Dept#’, Salary DECIMAL (7,2) BETWEEN 1 AND 99000.00 FORMAT ‘ZZ,ZZ9.99’) UNIQUE PRIMARY INDEX (EmpNo); Before inserting each row in the Employee table, Teradata FastLoad checks to verify that the value for: • Salary is in the range of 1 to 99000.00 • DeptNo is between -999 and 999 If it is known that the values for the DeptNo column are always in the range of -999 to 999, then they can improve the performance of the Teradata FastLoad job by removing the ZZ9 phrase from the CREATE TABLE statement in the Teradata FastLoad job script. Record Mode Load Anomaly When loading data in Record Mode into a NULLABLE DATE field, if the source data is a binary integer of value zero, then the Teradata Database sets the field to NULL, not to zero. UNIX Signals If Teradata FastLoad is running in a UNIX operating system, be aware of the UNIX signals used by Teradata FastLoad. Teradata FastLoad UNIX signals cannot be used in any module or routine programmed for use with Teradata FastLoad. Doing so causes an error in Teradata FastLoad. Teradata FastLoad uses the following UNIX signals: • SIGINT (interrupt signal) • SIGQUIT (quit signal) • SIGTERM (terminate signal) • SIGUSR1 (user signal 1) Teradata FastLoad Reference 61 Chapter 2: Using Teradata FastLoad Programming Considerations Note: Signals are predefined messages sent between two UNIX OS processes to communicate the occurrence of unexpected external events, or exceptions. Aborting a Teradata FastLoad session while Teradata FastLoad is in the middle of processing a job is an example of an exception. In this scenario, Teradata FastLoad uses the UNIX signals to trap the abort command, disconnect all sessions, do any necessary cleanup, and then terminate in an orderly manner. Restrictions and Limitations The following sections describe restrictions and limitations relevant for programming with the Teradata FastLoad utility. Session Limits The value specified with the SESSIONS command is not the only factor that limits the number of sessions that Teradata FastLoad establishes with the Teradata Database. The other limiting factors are: • The Teradata Database limit of one session per AMP. • The platform limit on the maximum number of sessions per application. This value is defined in the Communications Processor (COP) Interface software file, CLISPB.DAT, under the MaxSess variable. The TDP SET MAXSESSIONS command can be used to specify a platform limit. The default limit is equal to the server MAXSESS. • The limit of the network protocol software on network-attached systems. When invoking Teradata FastLoad, the actual session limit is determined by whichever limiting factor is encountered first. Space Requirements and Limitations Always estimate the final size of the Teradata FastLoad table, and make sure that the destination database on the Teradata Database has enough space to accommodate the Teradata FastLoad job. If the database that owns the Teradata FastLoad table or the error tables runs out of space, the Teradata Database returns an error message and Teradata FastLoad pauses your Teradata FastLoad job. When this happens, allocate more space to the database before restarting the job. Join Index Restrictions Teradata FastLoad does not maintain join indexes. Teradata FastLoad cannot be used to load data to tables with an associated join index on a Teradata Database. In this case, first drop the join index, then recreate it after running the Teradata FastLoad job. Note: Teradata FastLoad does not support hash and join indexes. Foreign Key References Limitation Teradata FastLoad does not support foreign key references in target tables. Attempting a Teradata FastLoad task or any action against a target table defined with a foreign key constraint produces an error condition. 62 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Programming Considerations Secondary Indexes Limitation Teradata FastLoad does not support target tables defined with secondary indexes. Attempting a Teradata FastLoad task against a target table defined with secondary indexes produces an error condition. To load such a table, first drop the secondary indexes. Then load the table and recreate the secondary indexes. Or, alternatively, if only nonunique secondary indexes (NUSI) are involved, consider using MultiLoad. Note: Teradata FastLoad does not support secondary indexes of any kind. Referential Integrity Limitation Attempting a Teradata FastLoad task or any action against tables with [referential integrity|defined triggers] produces an error condition. Defined Triggers Limitation Attempting a Teradata FastLoad task or any action against tables with [referential integrity|defined triggers] produces an error condition. Normalized Table Restrictions Teradata FastLoad is not allowed on a normalized PI table when PI column is part of the ignore column list. Teradata FastLoad is not allowed on a normalized NoPI table. Loading No Primary Index Tables A NoPI Table is a table that has no primary index. These tables can be used as staging tables where data is always appended to the table, making population of the table generally faster than that of a traditional table containing a primary index. Note: FastLoad is generally faster than TD TPump whether the target table is PI or NoPI. TD TPump has a bigger performance improvement with NoPI but it is still slower than FastLoad. The “table must be empty” restriction applies if the target table is a NoPI table. FastLoad cannot load into a populated NoPI table. See the No Primary Index discussion in SQL Data Definition Language (B035-1144). FastLoad can load Primary Index (PI) tables defined as MULTISET, but all duplicate rows are discarded, the target table is treated as if it were a SET table. NoPI tables are inherently MULTISET, since no duplicate row checking is possible (duplicate rows can be on different AMPs). Therefore, when FastLoad targets a NoPI table, no check is made for duplicate rows, and duplicate rows are not discarded. Conventions for Quotes Strings that contain single quotes or double quotes are not allowed to span records (lines); therefore, ensure that all quoted strings are limited to a single line. Teradata FastLoad Reference 63 Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines INMOD and Notify Exit Routines This section provides information about how to use input modification (INMOD) and notify exit routines. Overview This section describes the different types of routines and when to use them. INMOD Routines The term INMOD is an acronym for input modification routines. These are user exit routines that Teradata FastLoad and other load/export utilities can call to provide enhanced processing functions on input records before they are sent to the Teradata Database. When an INMOD routine is specified in the DEFINE command, it is the named routine, rather than a data source, that provides the input data records that Teradata FastLoad loads into the Teradata FastLoad table on the Teradata Database. Use INMOD routines to: • Select and validate data records before passing them to the Teradata Database • Convert fields in data records before passing them to the Teradata Database • Read data directly from different database system data sets, such as IMS, TOTAL, and so on, and create and pass a consolidated data record to the Teradata Database without using an intermediate tape or disk data set. Notify Exit Routines A notify exit routine specifies a predefined action to be performed whenever certain significant events occur during a Teradata FastLoad job. Notify exit routines are especially useful in operator-free environments where job scheduling relies heavily on automation to optimize system performance. For example, by writing an exit routine in C (without using CLIv2) and using the NOTIFY command, a routine to detect whether a Teradata FastLoad job succeeds or fails, how many records were loaded, what the return code is for a failed job, and so on can be provided. Programming Considerations for Routines This section describes programming languages supported for each type of routine, as well as other related considerations. Programming Languages Teradata FastLoad is written in: 64 • IBM C for mainframe-attached z/OS client systems • C for network-attached UNIX and Windows client systems Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines In all cases, INMOD and notify exit routines are dynamically loaded at run time, rather than link-edited into the Teradata FastLoad module. The programming languages listed in Table 13 can be used to write INMOD and notify exit routines, depending on the platform that runs Teradata FastLoad. Table 13 lists the programming languages supported. Table 13: Programming Languages Supported by Platform and Type of User-Developed Routine Platform INMOD Routine Notify Exit Routine Write z/OS IBM C, Assembler, COBOL, or PL/I IBM C • INMOD routines in IBM C • Notify exit routines in IBM C UNIX OS, Windows C C • INMOD and notify exit routines in C Note: Although it is neither certified nor supported, INMOD routines in COBOL can be written on network-attached client systems if the Micro Focus COBOL for UNIX compiler is used. However, INMOD and OUTMOD routines written in C/C++ are strongly recommended. Programming Structure Table 14 defines the structure, by programming language, for communicating between Teradata FastLoad and an INMOD or notify exit routines. Table 14: Programming Structures for Teradata FastLoad/INMOD Communication Routine Language Programming Structure C struct { long Status; long RecordLength; char buffer[32004]; } COBOL 01 INMOD-RECORD. 03 RETURN-CODE PIC S9(9) COMP. 03 RECORD-LENGTH PIC S9(9) COMP. 03 RECORD-BODY PIC X(32004). In each structure, the records must be constructed so that the left-to-right order of the data field corresponds to the order of the field names specified in the DEFINE command. Routine Entry Points Table 15 shows the entry points for INMOD routines. Teradata FastLoad Reference 65 Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Table 15: Entry Points for INMOD Routines Routine Language Entry Point IBM C on z/OS platforms dynamn C on all supported workstation platforms BLKEXIT COBOL and PL/I BLKEXIT Addressing Mode on z/OS Systems Use either 31-bit or 24-bit addressing for INMOD routines on mainframe-attached systems. The 31-bit mode provides access to more memory, which enhances performance for Teradata FastLoad jobs with a large number of sessions. Use the following linkage parameters to specify the addressing mode when building INMOD routines for z/OS systems: • AMODE(31) for 31-bit addressing • AMODE(24) for 24-bit addressing Teradata FastLoad/INMOD Routine Interface Teradata FastLoad exchanges information with an INMOD routine by passing an address that points to a three-value structure: status code, length, and body. Status Code Status code is the 32-bit signed binary value that carries information in both directions. Table 16 explains the six status codes used by the Teradata FastLoad-to-INMOD interface. Table 16: Teradata FastLoad--to--INMOD Interface Status Codes Value Description 0 Teradata FastLoad is calling for the first time. Teradata FastLoad expects the INMOD routine to return a data record. Note: At this point, the INMOD routine should perform its initialization tasks before sending a data record to Teradata FastLoad. 1 Teradata FastLoad is calling, not for the first time. Teradata FastLoad expects the INMOD routine to return a data record. 2 The client system has been restarted. The INMOD routine should reposition to the last checkpoint. Teradata FastLoad is not expecting the INMOD routine to return a record. Note: This is a one-time call, and Teradata FastLoad does not issue a subsequent call with a status code value of zero. 66 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Table 16: Teradata FastLoad--to--INMOD Interface Status Codes (continued) Value Description 3 A checkpoint has been written. The INMOD routine should remember the checkpoint position. Teradata FastLoad does not expect the INMOD routine to return a record. 4 The Teradata Database has failed. The INMOD routine should reposition to the last checkpoint. Teradata FastLoad is not expecting the INMOD routine to return a record. Note: This is a one-time call, and Teradata FastLoad does not issue a subsequent call with a status code value of zero. 5 The Teradata FastLoad job has ended. The INMOD routine should perform any required cleanup tasks. Note: This condition only applies to network-attached client systems. Table 17 describes the two status codes used by the INMOD-to-Teradata FastLoad interface. Table 17: INMOD-to-Teradata FastLoad Interface Status Codes Status Code Description 0 a record is being returned as the Body value any nonzero value the INMOD routine is at an end-of-file condition Length Length is a 32-bit signed binary value that the INMOD routine uses to specify the length, in bytes, of the data record. The INMOD routine can use a length value of zero to indicate an end-of-file condition. Body Body is the area where the INMOD routine places the data record. The maximum record length depends on the release/version level of the Teradata Database. Teradata FastLoad neither checks nor enforces record length restrictions on data provided by the INMOD routine. The record length, therefore, must not exceed the capability of the Teradata Database. Caution: To prevent data corruption, INMOD routines that cannot comply with these protocols should terminate if they encounter a restart code 2, 3, or 4. To support proper Teradata FastLoad restart operations, INMOD routines must save and restore checkpoint information as described here. If the INMOD saves checkpoint information in some other manner, a subsequent restart/recovery operation could result in data loss or corruption. Teradata FastLoad Reference 67 Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Teradata FastLoad/Notify Exit Routine Interface Teradata FastLoad accumulates operational information about specific events that occur during a Teradata FastLoad job. If the Teradata FastLoad job script includes a NOTIFY command with an EXIT option specification, Teradata FastLoad calls the named notify exit routine and passes to it when the specific events occur: • An event code to identify the event • Specific information about the event Table 18 lists the event codes and describes the data that Teradata FastLoad passes to the notify exit routine for each event. For a description of the events associated with each level of notification (low, medium, and high), see “NOTIFY” on page 137. Note: To support future enhancements, always ensure that notify exit routines ignore invalid or undefined event codes, and that they do not cause Teradata FastLoad to terminate abnormally. Note: Beginning with the release of Teradata Tools and Utilities 14.10, Teradata FastLoad supports 8-byte row counters. To display 8-byte counters in Notify events, use the new event with 64 as described in Table 18 and use the keyword EXIT64 rather than EXIT in the NOTIFY command. Note: Beginning with the release of Teradata Tools and Utilities 14.10, Teradata FastLoad supports the EON feature. To display large object names in Notify events, use the new event with EON as described in Table 18 and use the keyword EXITEON rather than EXIT in the NOTIFY command. The keyword EXITEON automatically supports the keyword EXIT64. Table 18: Events Passed to the Notify Exit Routine Event Code Event Event Description Data Passed to the Notify Exit Routine 0 Initialize Successful processing of the NOTIFY command • Version ID length—4-byte unsigned integer • Version ID string—32-character (maximum) array • Utility ID—4-byte unsigned integer • Utility name length—4-byte unsigned integer • Utility name string—32-character (maximum) array • User name length—4-byte unsigned integer • User name string—64-character (maximum) array • Optional string length—4-byte unsigned integer • Optional string—80-character (maximum) array 1 2 68 File or INMOD open Phase 1 begin Successful processing of the DEFINE command that specifies the file or INMOD routine name. • File name length—4-byte unsigned integer • File name—256-character (maximum) array The beginning of the insert • Table name length—4-byte unsigned integer phase, where the table name is • Table name—128-character (maximum) array specified by the INSERT • Database name—90 byte character array statement. Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Table 18: Events Passed to the Notify Exit Routine (continued) Event Code Event Event Description Data Passed to the Notify Exit Routine 3 Checkpoint Checkpoint information has been written to the restart log table. • Record number—4-byte unsigned integer 4 Phase 1 end The CHECKPOINT LOADING END request has successfully completed after the end of the insert phase. • Records read—4-byte unsigned integer • Records skipped—4-byte unsigned integer • Records rejected—4-byte unsigned integer • Records sent to the Teradata Database—4-byte unsigned integer 5 Phase 2 begin The END LOADING command is about to be sent to the Teradata Database. No data accompanies the phase 2 begin event code 6 Phase 2 end Processing of the END LOADING command completed successfully. • Records loaded—4-byte unsigned integer 7 Error table 1 Processing of the SEL COUNT(*) request completed successfully for the first error table. • Number of rows—4-byte unsigned integer 8 Error table 2 Processing of the SEL COUNT(*) request completed successfully for the second error table. • Number of rows—4-byte unsigned integer 9 Teradata Database restart Teradata FastLoad received a No data accompanies the Teradata Database restart event crash message from the code Teradata Database or from the CLIv2. 10 CLIv2 error Teradata FastLoad received a • Error code—4-byte unsigned integer CLIv2 error. 11 Teradata Database error Teradata FastLoad received a • Error code—4-byte unsigned integer Teradata Database error that will produce an exit code of 12. Note: Not all Teradata Database errors cause this event. An Error 3807, for example, while trying to drop or create a table does not terminate Teradata FastLoad. 12 Exit Teradata FastLoad Reference FastLoad is terminating. • Exit code—4-byte unsigned integer 69 Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Table 18: Events Passed to the Notify Exit Routine (continued) Event Code Event Event Description Data Passed to the Notify Exit Routine 13 Check point 64 Checkpoint information has been written to the restart log table. • Record number—8-byte unsigned integer been written to the restart log table. 14 Phase I end 64 The CHECKPOINT LOADING END request has successfully completed after the end of the insert phase. • • • • 15 Phase 2 end 64 Processing of the END LOADING command completed successfully. • Records loaded—8-byte unsigned integer 16 Error Table 1 64 Processing of the SEL COUNT(*) request completed successfully for the first error table. • Number of rows—8-byte unsigned integer 17 Error Table 2 64 Processing of the SEL COUNT(*) request completed successfully for the second error table. • Number of rows—8-byte unsigned integer 18 Initialize EON Successful processing of the • Version ID length—4-byte unsigned integer NOTIFY command • Version ID string—32-character (maximum) array • • • • • • • 19 Phase 1 begin EON Records read—8-byte unsigned integer Records skipped—8-byte unsigned integer Records rejected—8-byte unsigned integer Records sent to the Teradata Database—8-byte unsigned integer Utility ID—4-byte unsigned integer Utility name length—4-byte unsigned integer Utility name string—32-character (maximum) array User name length—4-byte unsigned integer User name string—a character pointer Optional string length—4-byte unsigned integer Optional string—80-character (maximum) array The beginning of the insert • Table name length—4-byte unsigned integer phase, where the table name is • Table name—a character pointer specified by the INSERT • Database name—a character pointer statement. Teradata FastLoad Sample INMOD Routines Teradata FastLoad software includes two sample INMOD routines that demonstrate how to write an INMOD routine using the C programming language. Table 19 describes these sample routines. 70 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad INMOD and Notify Exit Routines Table 19: Sample INMOD Routines Routine Description BLKEXITR.C Retrieves records from a data source and supplies them to Teradata FastLoad. It runs continuously until it reaches EOF. Note: The BLKEXITR.C sample INMOD routine supports Teradata FastLoad restart operations. BLKEXIT.C Generates data internally and passes the records to a Teradata FastLoad job for processing. To use this sample INMOD routine, supply a Teradata FastLoad job to load the data to the Teradata Database. Note: The BLKEXIT.C sample INMOD routine does not support Teradata FastLoad restart operations. The next two subsections provide sample Teradata FastLoad job scripts for calling the sample Teradata FastLoad INMOD routines. As required, each job script includes a Teradata FastLoad DEFINE command that specifies the INMOD option. These jobs can execute either interactively or in batch, as described earlier in this chapter. For a complete listing of each sample INMOD routine, see Appendix C: “INMOD and Notify Exit Routine Examples.” Calling BLKEXIT.C The following Teradata FastLoad job script calls the BLKEXIT.C sample INMOD routine: SESSIONS 1 ; LOGON tdpid/username,password ; DROP TABLE Fastest ; DROP TABLE Fasterr1 ; DROP TABLE Fasterr2 ; CREATE TABLE FastTest (Column1 Integer Format ’9(5)’ NOT NULL BETWEEN 1 AND 9999) UNIQUE PRIMARY INDEX (Column1) ; DEFINE Test(Integer) INMOD=Blkexit ; BEGIN LOADING FastTest ERRORFILES Fasterr1, Fasterr2 ; INSERT INTO FastTest (Column1) VALUES (:test); END LOADING ; LOGOFF ; Calling BLKEXITR.C The following Teradata FastLoad job script calls the BLKEXITR.C sample INMOD routine: * use your own account and password here. * LOGON sia1/weekly, weekly; DROP TABLE Error_1; DROP TABLE Error_2; DROP TABLE BlkExit; CREATE TABLE BlkExit Counter(Integer), c2(smallint), Teradata FastLoad Reference 71 Chapter 2: Using Teradata FastLoad Teradata FastLoad Job Scripts c3(integer), c4(smallint), c5(integer) UNIQUE PRIMARY INDEX (Counter); BEGIN LOADING BlkExit ErrorFiles Error_1, Error_2; DEFINE Counter(Integer), c2(smallint), c3(integer), c4(smallint), c5(integer) INMOD = BLKEXIT; INSERT INTO BlkExit (Counter, c2, c3, c4, c5 ) VALUES (:Counter, :c2, :c3, :c4, :c5 ); END LOADING; LOGOFF; Creating Custom INMOD Routines To meet specific system needs, either write custom INMOD routines, or modify the sample Teradata FastLoad INMOD routines to: • Select only records that meet specific criteria • Convert certain fields to a different data type • Perform other functions, as required Whenever a new INMOD routine is created, or modify an existing the new or modified routine must be compiled and linked into a shared object for use by Teradata FastLoad. Note for UNIX operating system users: Teradata FastLoad 6.0 and later uses dynamic linking to load INMOD routines at run time. As a result, any INMOD routines created or modified under earlier versions of the utility must be recompiled and re-linked. For procedures and examples of compiling and linking INMOD routines, see Appendix C: “INMOD and Notify Exit Routine Examples.” Teradata FastLoad Job Scripts This section describes the contents of the Teradata FastLoad job script and explains how to write one. 72 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Teradata FastLoad Job Scripts Definition A Teradata FastLoad job script, or program, is a set of Teradata FastLoad commands and Teradata SQL statements that actually load the data from the input data source or INMOD routine into the Teradata FastLoad table on the Teradata Database. Before running Teradata FastLoad in batch mode, a standard input file must be created that contains the Teradata FastLoad job script. Then use the appropriate input file specification to identify the standard input file when invoking Teradata FastLoad: • < infilename specification on network-attached client systems • SYSIN DDNAME specification on mainframe-attached z/OS client systems Enter Teradata FastLoad Commands Teradata FastLoad accepts the Teradata FastLoad commands and a subset of Teradata SQL statements described in Chapter 3: “Teradata FastLoad Commands.” Though the command keywords are all shown in uppercase characters in the syntax diagrams, the commands are not case sensitive. Use either uppercase or lowercase letters when typing the commands. When running Teradata FastLoad in interactive mode, wait for the Teradata FastLoad command prompt before entering a Teradata FastLoad command: FastLoad - Enter your command: Teradata FastLoad Job Script Example The following example Teradata FastLoad job script provides an overview of a typical Teradata FastLoad operation: SESSIONS 4; RECORD 100 THRU 100000; ERRLIMIT 25; LOGON tdpid/userid,password DROP TABLE FastTable; DROP TABLE Error1; DROP TABLE Error2; CREATE TABLE FastTable, NO FALLBACK ( ID INTEGER, UFACTOR INTEGER, MISC CHAR(42)) PRIMARY INDEX(ID); DEFINE ID (INTEGER), UFACTOR (INTEGER), MISC (CHAR(42)) FILE=FileName; SHOW; BEGIN LOADING FastTable ERRORFILES Error1,Error2 CHECKPOINT 10000; INSERT INTO FastTable (ID, UFACTOR, MISC) VALUES (:ID, :MISC); END LOADING; LOGOFF; Table 20 describes the commands used in this sample script. Teradata FastLoad Reference 73 Chapter 2: Using Teradata FastLoad Teradata FastLoad Job Scripts Table 20: FastLoad Entering Commands Command Description SESSIONS Directs Teradata FastLoad to log on to the Teradata Database for up to four sessions. RECORD Directs Teradata FastLoad to begin reading data at record 100 in the input data source and stop reading records at record 100,000. ERRLIMIT Directs Teradata FastLoad to stop processing when 25 errors occur. LOGON Logs the specified user on to the Teradata Database for up to four sessions, as specified in the SESSIONS command. DROP TABLE Makes sure that the Teradata FastLoad table and the two error tables do not already exist on the Teradata Database. Teradata FastLoad will not run if the two error tables exist from a prior job. And, though the Teradata FastLoad table can be an existing table, it must be empty. Thus, instead of deleting an existing Teradata FastLoad table, use the DELETE statement to remove all the rows. CREATE TABLE Creates the Teradata FastLoad table on the Teradata Database. DEFINE Defines the data fields in each record and identifies the input data source. This command corresponds to a Teradata SQL USING clause. SHOW Displays the active definitions for the input data source and the field names that were specified in the previous DEFINE command. This command allows the exact definitions in effect during the Teradata FastLoad operation to be verified. BEGIN LOADING Begins the loading phase of the Teradata FastLoad job. This command specifies the name of the Teradata FastLoad table and the two error tables, and, in this example, specifies that a checkpoint be taken every 10,000 records. INSERT Sends input data records to the Teradata Database and inserts rows into the Teradata FastLoad table. Teradata FastLoad processes the data records by: 1 Packaging them into large data blocks 2 Transferring them to the Teradata Database, where they are distributed to the AMPs END LOADING Directs the Teradata Database to redistribute (hash) the rows of data on the AMPs and store them in the Teradata FastLoad table. Upon successful completion, Teradata FastLoad returns a status message that displays the total number of: • Records read • Records skipped • Records sent to the Teradata Database • Records inserted as rows in the Teradata FastLoad table • Error records in the two error tables • Duplicate rows 74 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Run Multifile Teradata FastLoad Jobs Table 20: FastLoad Entering Commands (continued) Command Description LOGOFF Logs off all Teradata FastLoad sessions, terminates Teradata FastLoad, and presents the system command prompt. Run Multifile Teradata FastLoad Jobs This section covers the things to consider when running a multifile Teradata FastLoad job. A multifile Teradata FastLoad job is one that loads the Teradata FastLoad table with input data from more than one source. Do this by: 1 Using a LOGOFF command, with no END LOADING command, to intentionally pause the Teradata FastLoad job after the job has been initiated and loaded the data from the first source. 2 Successively restarting and pausing the Teradata FastLoad job to load the data from each subsequent input source. 3 Using an END LOADING command to terminate the Teradata FastLoad job after the data from the last input source has been loaded. Note: When running a multifile Teradata FastLoad job, the Teradata FastLoad table and the two error tables remain locked and are not available to users until the END LOADING command is used to conclude the Teradata FastLoad job. The following subsections provide example Teradata FastLoad job scripts that show how to load a table called Fast_Table with data stored in three different input data sources (FirstFile, SecondFile, and ThirdFile). “Command Functions” on page 76 describes each of the commands in the three job scripts. For a complete example using all loading commands, see Appendix C: “INMOD and Notify Exit Routine Examples.” Initiate the Teradata FastLoad Job and Loading the First Data Source The following Teradata FastLoad job script begins the multifile Teradata FastLoad job and loads data from the data source named FirstFile into the Teradata FastLoad table: LOGON tdpid/jwt,smart ; DROP TABLE Fast_Table ; DROP TABLE Error_1 ; DROP TABLE Error_2 ; CREATE TABLE Fast_Table (col1 (char(5), col2(integer)) ; BEGIN LOADING Fast_Table ERRORFILES Error_1, Error_2 ; DEFINE Field_1 (char(5)), Field_2 (integer) FILE = FirstFile ; INSERT INTO Fast_Table (col1, col2) VALUES (:Field_1, :Field_2) ; LOGOFF ; Teradata FastLoad Reference 75 Chapter 2: Using Teradata FastLoad Run Multifile Teradata FastLoad Jobs Note: These examples do not use a RECORD command to specify a range of records in the input data sources. By default, they load the entire contents of each source into the Teradata FastLoad table. To specify a range of records, use a RECORD command before each INSERT statement in multifile Teradata FastLoad job scripts. Restart the Teradata FastLoad Job and Loading the Second Data Source The following Teradata FastLoad job script loads the Teradata FastLoad table shown in the previous example with data from the data source named SecondFile: LOGON tdpid/jwt,smart ; BEGIN LOADING Fast_Table ERRORFILES Error_1, Error_2 ; DEFINE Field_1 (char(5)), Field_2 (integer) FILE = SecondFile ; INSERT INTO Fast_Table (col1, col2) VALUES (:Field_1, :Field_2) ; LOGOFF ; Restart the Teradata FastLoad Job and Loading the Third Data Source The following Teradata FastLoad job script loads the Teradata FastLoad table shown in the previous examples with data from the data source named ThirdFile: LOGON tdpid/jwt,smart ; BEGIN LOADING Fast_Table ERRORFILES Error_1, Error_2 ; DEFINE Field_1 (char(5)), Field_2 (integer) FILE = ThirdFile ; INSERT INTO Fast_Table (col1, col2) VALUES (:Field_1, :Field_2) ; LOGOFF ; Terminate the Teradata FastLoad Job Sending the END LOADING command is the last step in a multifile Teradata FastLoad job. Include it in the job used to process the last input data source, or submit it as a separate job script. The following Teradata FastLoad job script issues the END LOADING command in a separate job script: LOGON tdpid/jwt,smart ; BEGIN LOADING Fast_Table ERRORFILES Error_1, Error_2 ; END LOADING ; LOGOFF ; Command Functions Table 21 describes the multifile Teradata FastLoad job scripts from the preceding example. 76 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Run Multifile Teradata FastLoad Jobs Table 21: Teradata FastLoad Terminating Commands Command Description BEGIN LOADING Specifies the name of the Teradata FastLoad table and the two error tables, and starts the loading phase of the Teradata FastLoad job After executing the BEGIN LOADING command in the Teradata FastLoad job script that initiates the job and loads the first data source, the Teradata FastLoad response signifies the start of a new job: BEGIN LOADING COMPLETE In subsequent multifile Teradata FastLoad job scripts, the Teradata FastLoad response is: FastLoad is continuing a multifile job. CREATE TABLE Creates a new Teradata FastLoad table, ensuring it is empty when the Teradata FastLoad job initiates DEFINE Identifies the field in each record to be inserted into the Teradata FastLoad table and specifies the name of the input data source The field names should be the same for all of the multifile Teradata FastLoad job scripts. The FILE= attribute, however, successively identifies a different source data source for each portion of the multifile Teradata FastLoad job. DROP TABLE Makes sure that the Teradata FastLoad table and the two error tables do not already exist on the Teradata Database when the Teradata FastLoad job initiates END LOADING Distributes all of the rows to the Teradata FastLoad table The following message indicates that the command executed successfully: END LOADING COMPLETE Note: At this point, the Teradata FastLoad table and the two error tables are unlocked and available to any user with the appropriate privileges. INSERT Transfers the data records from the specified input data source to the Teradata FastLoad table The Teradata Database automatically takes a checkpoint after the last record of each insert operation. Teradata FastLoad Reference 77 Chapter 2: Using Teradata FastLoad Handling Teradata FastLoad Errors Table 21: Teradata FastLoad Terminating Commands (continued) Command Description LOGOFF Pauses the Teradata FastLoad operation when executed before an END LOADING command, which is the case for each multifile Teradata FastLoad job script that identifies a new data source Because the LOGOFF command is entered during the loading phase, Teradata FastLoad responds with the message: FastLoad Paused. When a Teradata FastLoad job is paused during the loading phase, the tables remain locked and cannot be accessed until Teradata FastLoad executes the END LOADING command. When submitted after an END LOADING command, as in the last multifile Teradata FastLoad job script, the LOGOFF command terminates the Teradata FastLoad job. In this case, Teradata FastLoad displays: logoff; and an end-of-job status message, such as: **** 20:36:06 **** 20:36:11 Seconds' . . . **** 20:36:11 Logging off all sessions Total processor time used = '0.499203 Start : End : Highest FDL4818 Thu Sep 20 20:35:22 2012 Thu Sep 20 20:36:11 2012 return code encountered = '0'. FastLoad Terminated and then presents the system command prompt. LOGON Logs user JWT on to the Teradata Database Handling Teradata FastLoad Errors This section provides information about handling Teradata FastLoad errors. Teradata FastLoad Error Conditions While processing a Teradata FastLoad job script, Teradata FastLoad tracks and records information about five types of error conditions that cause the Teradata Database to reject an input data record. Table 22 describes these error conditions. Table 22: Error Conditions that Cause Teradata Database to Reject an Input Data Record 78 Error Conditions Description Constraint violations Records from the input file that do not comply with the range constraints that were specified in the CREATE TABLE statement. Conversion errors Records from the input file that fail a data type conversion that was specified in the DEFINE command. Duplicate rows Records from the input file that are exact duplicates of existing rows. Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Handling Teradata FastLoad Errors Table 22: Error Conditions that Cause Teradata Database to Reject an Input Data Record (continued) Error Conditions Description Unavailable AMP conditions Records from the input file that are destined for a nonfallback table on an AMP that is down. Unique primary index violations Records from the input file that have a value for the unique primary index field that already exists, but is not a duplicate row. Note: Teradata FastLoad does not store duplicate rows in an error table. Additionally, when operating in batch mode, Teradata FastLoad returns the system error codes for error conditions encountered during Teradata FastLoad operations. (Teradata FastLoad does not return system error codes when operating in interactive mode.) This subsection describes the procedures for handling the five types of Teradata FastLoad error conditions. See Messages (B035-1096) for more information about system error messages. Error Recording Teradata FastLoad stores the input data records related to constraint violations, conversion errors, unavailable AMP conditions, and unique primary index violations in the two error tables specified in the BEGIN LOADING command. Table 23 lists error tables. Table 23: Error Tables Table Stores Records That Produced These Error Types errortname1 • Constraint violations • Conversion errors • Unavailable AMP conditions These types of errors always occur during the loading phase of the Teradata FastLoad job after executing the BEGIN LOADING command, but before the END LOADING command. errortname2 Unique primary index violations This type of error always occurs during the end-loading phase of the Teradata FastLoad job after executing the END LOADING command. Teradata FastLoad discards all records that produce a duplicate row error, but includes the total number of duplicate rows encountered, along with the total records in each error table, in the end-of-job status report. Error Table Formats Table 24, the first error table, errortname1, has three columns: Teradata FastLoad Reference 79 Chapter 2: Using Teradata FastLoad Handling Teradata FastLoad Errors Table 24: errortname1 Columns Column Contains DataParcel Entire data record, as provided by the source producer operator DataParcel is used as the primary index for the first error table. The data record string can be up to 64,000 bytes, depending on which version of the DBS the job is run against. ErrorCode Teradata Database return code for the error condition, as specified in Messages (B035-1096) ErrorFieldName Name of the data item that caused the error condition, as specified in the fieldname attribute of the DEFINE command in the Teradata FastLoad job script Note: If the length of the data item name is greater than 120 characters, the DBS will truncate the data item name to the length of 120 characters. The format of the second error table, errortname2, is identical to that of the Teradata FastLoad target table. Correcting Errors Though the procedures are somewhat different, depending on the error table, correcting errors is a three-step process, where: 1 The error information is retrieved from the error tables on the Teradata Database. 2 Errors are evaluated and correct ed. 3 Corrected records are inserted into the Teradata FastLoad table. After the job has completed, another utility, such as BTEQ, must be used to access the Teradata Database, because Teradata FastLoad only operates on an empty table. The procedures and examples in the following subsections are performed under BTEQ. They assume BTEQ has been invoked and logged on to the Teradata Database. For more information about using BTEQ, see the Basic Teradata Query Reference (B035-2414). Procedure for Correcting Errors in the First Error Table Use the following procedure to correct errors recorded in the error table specified as errortname1: 1 Use the following Teradata SQL statement to retrieve the error code and field name for each error in the first error table: SELECT ErrorCode, ErrorFieldName FROM ErrorCode ; errortname1 ORDER BY where errortname1 is the name specified for the first error table. BTEQ responds with a list of the error codes and the associated field names, formatted as follows: ***Query completed. 2 rows found. 2 columns returned. ***Total elapsed time was 1 second. 80 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Handling Teradata FastLoad Errors ErrorCode --------2679 2679 ErrorFieldName -----------------A A The values listed in the ErrorCode column are the Teradata Database return codes for each error condition, as specified in Messages (B035-1096). The values listed in the ErrorFieldName column are the names of the fields that caused each error. 2 Use the following BTEQ commands and Teradata SQL statements to retrieve the data records for each error in the first error table and store them in the specified err.out file on the client system: • If the values in the ErrorCode column indicate that either a constraint violation or a conversion error occurred, retrieve the DataParcel information in record mode: .SET RECORDMODE ON .EXPORT DATA FILE=err.out SELECT DataParcel FROM errortname1 In record mode, the Teradata Database returns the SELECT statement results in client computer format, which is usually a hexadecimal dump. Thus, if all of the fields identified in the Teradata FastLoad DEFINE commands were also specified in the INSERT statements, the data retrieved from the Teradata FastLoad error table is in the same format as it was in the original input data source. • Otherwise, if the values in the ErrorCode column indicate that the errors were all caused by unavailable AMP conditions, do not use the RECORDMODE command: .EXPORT DATA FILE=err.out SELECT DataParcel FROM errortname1 Caution: 3 Use the ErrorCode and ErrorFieldName information returned in Step 1 and the DataParcel information returned in Step 2 to determine which records to correct and reload to the Teradata Database. The methods used to correct the individual error conditions will vary, depending on the number and types of errors. 4 After the errors have been corrected, use the following BTEQ commands and Teradata SQL statements to insert the corrected records into the Teradata FastLoad table on the Teradata Database. To Use the transfer the data to the Teradata Database BTEQ IMPORT command define the fields in each record Teradata SQL USING modifier insert a record into the Teradata FastLoad table Teradata SQL INSERT statement Do not reference the first two bytes in the INSERT statement for data records that were exported from the Teradata Database in record mode. Instead, make the first field (variable parameter) in the USING modifier a dummy SMALLINT field. When selecting data in record mode, the variable-length columns are all preceded by a two-byte field whose value indicates the length of the data field. But, because the Teradata FastLoad Reference 81 Chapter 2: Using Teradata FastLoad Handling Teradata FastLoad Errors DataParcel column of the errortname1 table is defined as a variable-length field, the first two bytes always indicate the length. If this field is not referenced in the INSERT statement, the Teradata Database ignores this portion of each record in the input data. 5 Repeat steps 2 through 4 as required to resolve all of the errortname1 error conditions. 6 Drop the errortname1 table from the Teradata Database after all of the errors have been resolved. Procedure for Correcting Errors in the Second Error Table Use the following procedure to correct errors recorded in the error table specified as errortname2: 1 Use the following Teradata SQL statement to retrieve all rows from the second error table: SELECT * FROM errortname2 ORDER BY cname ; where: Syntax Element Description cname Unique primary index for the table errortname2 Name of the second error table The BTEQ response is a list of the contents of the second error table, ordered by the values in the primary index column. 2 Use the following Teradata SQL statement to retrieve each row from the Teradata FastLoad table that has a primary index value identical to a row retrieved from the second error table: SELECT * FROM tname WHERE cname = errorvalue where: 3 4 82 Syntax Element Description cname Index of the Teradata FastLoad table errorvalue index value retrieved from the second error table tname Name of the Teradata FastLoad table Compare the rows selected from the error table with the rows selected from the Teradata FastLoad table and determine which is correct. • If the row selected from the error table is correct, then use a Teradata SQL DELETE statement to delete the incorrect row from the Teradata FastLoad table, and an INSERT statement to insert the correct row. • If the row selected from the Teradata FastLoad table is correct, then use a Teradata SQL DELETE statement to delete the corresponding row from the error table. Repeat Steps 2 and 3 until all rows in the error table are accounted for. Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad Handling RDBMS Retryable and Restartable Error Codes 5 Drop the errortname2 table from the Teradata Database after all errors have been resolved. Handling RDBMS Retryable and Restartable Error Codes Beginning with Teradata Tools and Utilities 14.10, RDBMS returns a list of retryable error codes: • When the user does not have permission on the SYSLIB.DBSRETRYABLEERRORS view, Teradata FastLoad displays the DBS “no permission” message and an information message that the internal list of retryable will be used for the job, and sets the job's return code to 4 (assuming there are no other errors for the rest of the job). • When Teradata FastLoad 14.10 runs with versions of the Teradata Database prior to 14.10, Teradata FastLoad displays “RDBMS error 3807: Object 'SYSLIB.DBSRETRYABLEERRORS' does not exist” and “The job will use its internal retryable error codes,” and the job's return code remains unchanged. • In the rare instance that zero rows are returned from the SYSLIB.DBSRETRYABLEERRORS view, FastLoad displays an information message that zero rows were returned and that the user that the internal list of retryable errors will be used for the job, and the job's return code remains unchanged. • When one or more rows are returned from the SYSLIB.DBSRETRYABLEERRORS view, FastLoad does not display any information about the DBS data-related error view, and the job's return code remains unchanged. For the detailed information regarding the list of retryable errors codes, see Utilities: Volume 1 (A-K)(B035-1102). FastLoad TMSM Integration Teradata Multi-System Manager (TMSM) is the monitoring and control facility for a variety of Dual Active Solutions offered by Teradata. The expected users of this facility are administrators of the Enterprise Data Warehouse (EDW) such as Teradata Database administrators, ETL maintenance staff, systems administrators, or anyone who needs to monitor and control processes that include, but are not limited to, Teradata Load and Unload Utilities, Teradata SQL, ETL tools, and Teradata Database. To integrate with TMSM, FastLoad has been enhanced to collect operational metadata and event data, obtain the Unit of Work ID (UOW ID) from TMSM for a job, and send such data to TMSM for monitoring purposes using the “send event” interface as described in the TMSM Event System API Reference. By default, a FastLoad job will send events to TMSM as long as TMSM is active. If TMSM is not active, the job will continue to run, except that no events will be sent to TMSM. Teradata FastLoad Reference 83 Chapter 2: Using Teradata FastLoad FastLoad Statistics Simple ETL process monitoring requires the tracking of the “start” and “end” of the process, which can include multiple “steps”, each of which represents an activity or event of the process to be monitored. In terms of FastLoad, a load job can be regarded as such a process. FastLoad follows the flow required by TMSM: 1 Obtain a system-generated UOW ID from TMSM for a FastLoad job 2 Send a “start” event to TMSM along with the UOW ID 3 Optionally send one or more “step” events to TMSM along with the UOW ID 4 Send an “end” event to TMSM along with the UOW ID Below is an example of how a FastLoad job sends event messages to TMSM in order for the job to be monitored: • At job start: “Connecting session(s)” • At step start of acquisition: “Acquisition begins” • At step checkpoint time: “Checkpoint completes” • At step end of acquisition: “Acquisition completes” • At step end of application: “Application complete” • At job end: “Job terminating” FastLoad Statistics Performance Statistics Teradata FastLoad displays: 1) At the end of the Acquisition Phase Acquisition Phase statistics: Elapsed time: 00:07:43 (hh:mm:ss) CPU time: 94.89 Second(s) MB/sec: 41.19 MB/cpusec: 200.99 The abbreviation "MB/sec"'s longhand is megabytes (Total data FastLoad sends) per elapsed second. The abbreviation "MB/cpusec"'s longhand is megabytes( Total data FastLoad sends) per cpu second. 2) At the end of the application phase: Application Phase statistics: Elapsed time: 00:02:43 (hh:mm:ss) These performance statistics could be varied on different platforms. An example of statistics output: FastLoad displays at the end of Acquisition: 84 Teradata FastLoad Reference Chapter 2: Using Teradata FastLoad FastLoad Statistics **** 11:53:09 Acquisition Phase statistics: Elapsed time: 00:00:13 (in hh:mm:ss) CPU time: 1.69 Seconds MB/sec: 4.25 MB/cpusec: 32.68 FastLoad displays at the end of Application Phass: **** 11:53:13 Application Phase statistics: Elapsed time: 00:00:04 (in hh:mm:ss) Teradata FastLoad Reference 85 Chapter 2: Using Teradata FastLoad FastLoad Statistics 86 Teradata FastLoad Reference CHAPTER 3 Teradata FastLoad Commands This chapter provides detailed descriptions of the Teradata FastLoad commands and the Teradata FastLoad version of the INSERT statement which can be executed from the Teradata FastLoad utility. Teradata FastLoad accepts both Teradata FastLoad commands and a subset of Teradata SQL statements. The Teradata FastLoad commands perform session control and data handling activities. The Teradata SQL statements define and manipulate the data stored in the Teradata Database. Experienced users can also refer to the simplified command descriptions in the Teradata Tools and Utilities Command Summary (B035-2401). This book provides the syntax diagrams for each Teradata client utility. Syntax Notes The following syntax rules apply when using Teradata FastLoad commands: • Commands may begin with a period, but do not have to begin with a period. • If there is no leading period, then there must be a semicolon at the end. • If the command has a leading period, it must all be on one line. Commands that begin with a period cannot span multiple lines. • Although a semicolon character should always be used to signify the end of a Teradata FastLoad command, it is not required for simple commands such as SHOW and SESSIONS. • The ending semicolon character is optional for Teradata FastLoad commands that both begin with a period and are specified on a single line. • Teradata SQL statements must not have a leading period and must always end with a semicolon. • If the command contains unbalanced double or single quotes, the command must be started with a period and on a single line. It can terminate with semi-colon (;). Object Name Restrictions The following Syntax rules apply to object names: • Teradata FastLoad Reference A semicolon cannot be used in an object name because a semicolon is used to designate the end of a Teradata FastLoad command. 87 Chapter 3: Teradata FastLoad Commands Syntax Notes • 30 bytes cannot be used as the quoted object name length for the database that does not support EON and 128 characters cannot be used as the quoted object name length for the database that supports EON. • Dictionary (UDD) object names cannot be used. See Appendix A: “How to Read Syntax Diagrams” for a description of how to read the syntax diagrams used in this book. Geospatial Data Restrictions The following rule applies to geospatial data: • Teradata FastLoad does not support ST_Geometry geospatial data. LOB Data Restrictions The following rule applies to LOB data: • Teradata FastLoad does not support LOB data. NUMBER Data Restrictions The following restrictions apply to NUMBER: 88 • Nullif and apply-where clause support includes NUMBER is not included. • SET and ACCEPT commands cannot assign and accept NUMBER. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands AXSMOD AXSMOD Purpose The AXSMOD command specifies the name and optional initialization string for an access module that provides data to the Teradata FastLoad utility on network-attached client systems. For more information about specific access modules, see the Teradata Tools and Utilities Access Module Reference (B035-2425). Syntax ; AXSMOD name "init-string" 2411A014 where Syntax Element Description init-string Optional initialization string for the access module. The initialization string can contain double quotes, but not single quotes. name Name of the access module file to be used to import data. These access modules include: • OLE DB Access Module (oledb_axsmod.dll on Windows platforms) • Named Pipes Access Module • Teradata WebSphere® MQ Access Module (client version) • Teradata WebSphere® MQ Access Module (server version) • Teradata Access Module for JMS (libjmsam.so on AIX, Solaris SPARC, Solaris Opteron, Linux and z/Linux platforms, libjmsam.sl on HP-UX pa RISC and HP-UX Itanium platforms, and libjmsam.dll on Windows platforms) See the Teradata Tools and Utilities Access Module Reference (B035-2425) for the name of the access module file for each platform. A personal shared library file name can be used if custom access module is used. The AXSMOD option is not required for importing disk files on either network-attached or mainframe-attached client systems, or magnetic tape files on mainframe-attached client systems. It is required for importing magnetic tape and other types of files on network-attached client systems. To specify the OLE DB Access Module, Named Pipes Access Module, or the WebSphere MQ Access Module for specific platforms, see the Teradata Tools and Utilities Access Module Reference (B035-2425). Teradata FastLoad Reference 89 Chapter 3: Teradata FastLoad Commands AXSMOD Usage Notes Table 25 describes the things to consider when using the AXSMOD command. Table 25: Usage Notes for AXSMOD Topic Usage Notes When to Use the AXSMOD Command The AXSMOD command is not required for loading: Command Placement • Disk files on either network or mainframe-attached client systems • Magnetic tape files on mainframe-attached client systems It is required for loading magnetic tape and other types of files on network-attached client systems. When using an access module, the AXSMOD command must be stated before the DEFINE command in the Teradata FastLoad job script. If the AXSMOD command is stated after the DEFINE command, Teradata FastLoad terminates with an error message. Example The following example provides the volume set name and owner as initialization parameters for the REELlibrarian access module, called libprmrl.so: AXSMOD libpmrl.so “-V FastLoad -O lmn” ; where 90 • libpmrl.so is the name of the access module • FastLoad is the name of the volume set • lmn is the owner of the volume set Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands BEGIN LOADING BEGIN LOADING Purpose The BEGIN LOADING command: • Identifies the Teradata FastLoad table to receive data transferred from a data source on the client computer. • Specifies the names of the two error tables. • Starts a new Teradata FastLoad job or restarts a job that has been paused. • Locks the tables specified in the command so that users cannot access them until an END LOADING command is executed. (After the end loading phase completes, any user with the appropriate privileges can access these tables.) • Optionally: • Indicates how often checkpoints are taken. • Identifies the presence of null indicators. Syntax BEGIN LOADING A ERRORFILES dbname. dbname. errortname1, dbname. tname1 A errortname2 B ; B NODROP CHECKPOINT integer DATAENCRYPTION value INDICATORS 2411C011 where Syntax Element Description dbname Name of the database in which each table resides. Teradata FastLoad uses the current default database if a database name is not included with the table name specifications. To refer to more than one database, include the database name with the table name specifications. tname1 Name of the Teradata FastLoad target table to receive the data from the client system. A name that duplicates the name of an existing table cannot be used unless restarting a paused Teradata FastLoad job. The name must be a new table name. Teradata FastLoad Reference 91 Chapter 3: Teradata FastLoad Commands BEGIN LOADING Syntax Element Description errortname1 Name of the first error table. A name that duplicates the name of an existing table cannot be used unless restarting a paused Teradata FastLoad job. The name must be a new table name. errortname2 Name of the second error table. A name that duplicates the name of an existing table cannot be used unless restarting a paused Teradata FastLoad job. The name must be a new table name. CHECKPOINT Enables the checkpoint option. integer Value that specifies the number of rows transmitted to the Teradata Database between checkpoints using the CHECKPOINT keyword to enable the checkpoint option. (The checkpoint option is not enabled if it is not included the CHECKPOINT keyword in the BEGIN LOADING command.) For more information about specifying the CHECKPOINT integer value, see Table 26. INDICATORS Places null indicator bits at the front of each record. The number of fields in each record determines how many bytes contain null indicators, as described in the Indicators topic under “Usage Notes” for this command. Note: The INDICATORS specification cannot be used when loading records in variable-length text format. If the Teradata FastLoad job script specifies INDICATORS in the BEGIN LOADING command and the VARTEXT option in the SET RECORD command, the utility terminates with an error message. Note: INDICATORS mode is not recommended when using TEXT record format. Please use UNFORMATTED record format instead. DATAENCRYPTION value Enables data encryption for a Teradata FastLoad job. The options for value are: • ON = The requests will be encrypted. • OFF = The requests will not be encrypted. This option will apply only to the BEGIN LOADING request and the requests after the BEGIN LOADING command. Using this option overwrites the data encryption settings specified by both the runtime parameters and in the floadcfg.dat configuration file. NODROP Retains the error table at the end of a job, even if the error table is empty. Caution: When using this option, be careful to manually drop all error tables before running additional jobs that use the same tables. Failure to do so results in Teradata Database errors. (Default) If this option is not specified, the error table is dropped at the end of a job if the error table is empty. 92 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands BEGIN LOADING Usage Notes Table 26 describes the things to consider when using the BEGIN LOADING command. Table 26: Usage Note for BEGIN LOADING Topic Usage Notes Required Privileges The user ID that is logged in to the Teradata FastLoad job must have: • SELECT and INSERT privileges on the Teradata FastLoad table • CREATE TABLE privilege on the database that owns the two error tables Restart Log Table To run Teradata FastLoad, the following privileges must be available to user PUBLIC on the Teradata FastLoad restart log table (SYSADMIN.FASTLOG): • • • • Error Tables Descriptions DELETE INSERT SELECT UPDATE Teradata FastLoad creates two error tables when executing the BEGIN LOADING command: • The error table specified by errortname1 contains records that were rejected because of an error other than unique primary index or duplicate row violation. • The error table specified by errortname2 contains records that violated the unique primary index constraint. Duplicate Records The Teradata Database ignores duplicate records, which are not inserted in either error table. Reusing Table Names If an error table has one or more rows, it is not dropped from the Teradata Database at the end of a Teradata FastLoad job. To reuse the names specified for the error tables, use the DROP TABLE statement to remove the tables from the Teradata Database. For more information, see “Error Recording” on page 79. Checkpoints The CHECKPOINT option defines points in a job where Teradata FastLoad pauses to record that the Teradata Database has processed a specified number of input records. When checkpoints are used, the entire Teradata FastLoad job need not be run if it stops before completion. Teradata FastLoad uses the checkpoint information in the restart log table to determine the restart location. When specifying the integer value for the CHECKPOINT option: • If zero is entered as the value, Teradata FastLoad processes the BEGIN LOADING command as if a CHECKPOINT value is not entered. • If a value is not entered, or if a value is entered that is not an integer, the Teradata Database returns a syntax error. For more information, see “Checkpoint Tradeoffs” on page 58. Teradata FastLoad Reference 93 Chapter 3: Teradata FastLoad Commands BEGIN LOADING Table 26: Usage Note for BEGIN LOADING (continued) Topic Usage Notes Indicators Indicators are bits at the beginning of a record that identify the nulled fields in the record. When INDICATORS in the BEGIN LOADING command are specified, Teradata FastLoad expects the first bytes of the record to contain an indicator bit for each record field. If the INDICATORS option is set but indicator bits are not entered at the beginning of the record, Teradata FastLoad assumes that the first field contains indicator bytes and loads the record incorrectly. Indicator bits must be stored in a minimum of eight-bit bytes. For example, if a record contains from one to eight fields, one byte is required for the indicator bits. If a record contains from nine to 16 fields, two bytes are required for the indicator bits, and so on. Set unused bits in indicator bytes to zero. Indicator Bit Positions The positions of the indicator bits correspond to the record fields. The first bit in the byte is the indicator for the first field in the record. If an indicator bit is set to 1, the Teradata Database nulls the corresponding field when the record is loaded. If the indicator bit is set to zero, the Teradata Database loads the data specified for that field. The following figure shows a record containing indicators. These fields are nulled 01001000 00000000 F1 F2 F3 F4 F5 F6 F7 F8 .... F16 2411A015 See the following documents for more information about indicator bits and the INDICATORS option: • Teradata Call-Level Interface Version 2 Reference for MainframeAttached Systems (B035-2417) • Teradata Call-Level Interface Version 2 Reference for Network-Attached Systems (B035-2418) Example The following command example starts a Teradata FastLoad job: BEGIN LOADING Employee ERRORFILES ErrTable1,ErrTable2 CHECKPOINT 50000; If the command is used to start a new job, then Teradata FastLoad responds: BEGIN LOADING COMPLETE! When BEGIN LOADING restarts a partially completed job, then Teradata FastLoad responds: FastLoad is continuing a paused job! If BEGIN LOADING is used to continue a multifile job, then Teradata FastLoad responds: FastLoad is continuing a multifile job! 94 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands CLEAR CLEAR Purpose The CLEAR command cancels the definitions that were specified by a previous DEFINE command, including the input data source. Syntax CLEAR ; 2411A016 Usage Notes The CLEAR command is: • Local to Teradata FastLoad • Not transmitted to the Teradata Database • Intended for online use Example The following command example cancels the field definitions and input data source or INMOD name of a previous DEFINE command: CLEAR ; Completion Message The Teradata FastLoad completion message is: Warning: All previous column and file definitions cleared! If the SHOW command now is entered, the result is blank. Teradata FastLoad Reference 95 Chapter 3: Teradata FastLoad Commands DATEFORM DATEFORM Purpose The DATEFORM command specifies the form of the DATE data type specifications for the Teradata FastLoad job. Syntax DATEFORM INTEGERDATE ANSIDATE ; 2411A017 where Syntax Element Description INTEGERDATE Keyword that specifies integer DATE data types for the Teradata FastLoad job. This is the default Teradata DATE data type specification for Teradata FastLoad jobs if a DATEFORM command is not entered. ANSIDATE Keyword that specifies ANSI fixed-length CHAR(10) DATE data types for the Teradata FastLoad job. Usage Notes Table 27 describes the things to consider when using the DATEFORM command. Table 27: Usage Notes for DATEFORM Topic Usage Notes Command Placement and Frequency Only use one DATEFORM command. Data Type Conversions When the ANSIDATE specification is used, ANSI/SQL DateTime data types must be converted to fixed-length CHAR data types when specifying the column/field names in the Teradata FastLoad DEFINE command. The command must be entered before the LOGON command. See Table 28 for a description of the fixed-length CHAR representations for each DATE, TIME, TIMESTAMP, and INTERVAL data type specification. Release Applicability 96 The ANSIDATE specification is valid for Teradata FastLoad jobs. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE DEFINE Purpose The DEFINE command: • Describes the fields in a record of input data that are inserted in the Teradata FastLoad table • Identifies the name of the input data source or use of an INMOD routine Teradata FastLoad translates the DEFINE command specifications into a Teradata SQL USING clause, and links the USING clause with a subsequent INSERT statement. The Teradata FastLoad job must include DEFINE specifications for each field in a record from the input data source before executing an INSERT statement. Note: Though every field used in an INSERT statement must have been previously defined in a DEFINE command, every field so defined need not be used in an INSERT statement. Teradata FastLoad also uses the DEFINE command data type specifications to determine the format and record length of stored data. Syntax A DEFINE , DEF fieldname (datatype ) ,NULLIF value = ; A = filename FILE DDNAME INMOD = name 2411A018 where Syntax Element Description fieldname Name of a field in a record of the input data source, from left to right. You cannot use FILE, DDNAME, or INMOD for a fieldname. Note: Some or all of the field names can be omitted if the accompanying INSERT statement uses the “wild card” table name specification (tname.*) for all of the columns in the table. For the syntax of the tname.* specification, see the INSERT statement description later in this chapter. Teradata FastLoad Reference 97 Chapter 3: Teradata FastLoad Commands DEFINE Syntax Element Description datatype Keyword or keyword phrase that specifies the data type of the field. For details on data types and data conversions, see SQL Data Types and Literals (B035-1143). The valid data types for records in a Teradata FastLoad table are: BIGINT BYTE (n) BYTEINT CHARACTERS (n) DATE DECIMAL (x) or DECIMAL (x,y) FLOAT GRAPHIC (n) INTEGER LONG VARCHAR LONG VARGRAPHIC NUMBER (p) NUMBER NUMBER(p,s) NUMBER(*,s) PERIOD(DATE) PERIOD(TIME[(n)]) PERIOD(TIME[(n)] WITH TIME ZONE) PERIOD(TIMESTAMP[(n)]) PERIOD(TIMESTAMP[(n)] WITH TIME ZONE) SMALLINT VARBYTE (n) VARCHAR (n) VARGRAPHIC (n) NULLIF=value Keyword phrase that loads the Teradata Database field with a null value if the defined client field contains the specified value. The value specification can be 1 to 80 bytes in length, and it pertains only to BYTE, CHAR and GRAPHIC types. It does not pertain to integer and float data types. The NULLIF option occurs only with the DEFINE command. 98 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Syntax Element Description FILE=filename Keyword phrase specifying the name of the data source that contains the input data. fileid must refer to a regular file. Specifically, pipes are not supported. Note: For mainframe-attached systems, replace FILE with DDNAME for z/OS. DDNAME is the sequential data set for z/OS. Note: In UNIX and Windows, the fileid FastLoad supports file name of size of up to 1024.The filename can contain the following characters: parenthesis, comma, or equal sign. INMOD=name Keyword phrase specifying the name of a user exit routine that provides input data records. For more information about using INMOD routines, see “INMOD and Notify Exit Routines” on page 64. Note: If INMOD module output messages to stdout, the character set that INMOD uses is independent of the character set that Teradata FastLoad uses; the display on stdout can be of mixed character sets. For example, IMMOD can output messages in ASCII and Teradata FastLoad can output messages in UTF-16. Note: In UNIX and Windows, FastLoad supports the INMOD name of size of up to 1024. The INMOD name can contain the following characters: parens, comma, or equal sign. Usage Notes The following table describes the things to consider when using the DEFINE command. • Data Type Descriptions Depending on the data type phrase of the CREATE TABLE statement, Teradata FastLoad stores data with CHAR(n) and VARCHAR(n) data type specifications as follows: • • • When the DEFINE command data type attribute is CHAR(n) and the: • CREATE TABLE datadesc attribute is CHAR(n), Teradata FastLoad stores the data in fixed-length format, entire field. • CREATE TABLE datadesc attribute is CHAR(m), Teradata FastLoad stores the data in fixed-length format, padded if m > n, truncated if m < n. • CREATE TABLE datadesc attribute is VARCHAR(m), Teradata FastLoad stores the data in variable-length format with blanks trimmed. When the DEFINE command datatype attribute is VARCHAR(n) and the: • CREATE TABLE datadesc attribute is VARCHAR(m), Teradata FastLoad stores the data in variable-length format, no padding, blanks not trimmed. • CREATE TABLE datadesc attribute is CHAR(m), Teradata FastLoad stores the data in padded or truncated format, as required. Input Length and Field Descriptions Table 28 lists the input length and field description for each data type specification. Teradata FastLoad Reference 99 Chapter 3: Teradata FastLoad Commands DEFINE Note: In a UTF-16 session, a character size is 2 bytes. For CHAR(n) or VARCHAR(n) field of the CREATE TABLE, the corresponding field size in DEFINE command must be double, that is CHAR(n*2) or VARCHAR(n*2) respectively. Note: In a UTF-8 session, a character size is from 1 to 3 bytes. For CHAR(n) or VARCHAR(n) field of the CREATE TABLE, the corresponding field size in DEFINE command must be triple, that is CHAR(n*3) or VARCHAR(n*3) respectively. Table 28: Input Length and Field Descriptions Data Type Length Description BIGINT 8 bytes 64-bit signed binary BYTE(n) n bytes n bytes BYTEINT 1 byte 8-bit signed binary CHAR, CHARS(n), and CHARACTERS(n) n bytes n ASCII characters DATE 4 bytes 32-bit integer in YYYMMDD format as a decimal value Note: If a DATEFORM command has been used to specify ANSIDATE as the DATE data type, Teradata FastLoad internally converts each DATE field to a CHAR(10) field. DECIMAL(x) and DECIMAL(x,y) 1, 2, 4, 8, or 16 bytes for network; packed decimal for mainframe 128-bit double precision, floating point For more information on the DECIMAL data type, see SQL Data Types and Literals (B035-1143). FLOAT 8 bytes 64-bit, double precision, floating point GEOSPATIAL DATA maximum 64000 FastLoad does not support Geospatial data represented by LOBs. INTEGER 4 bytes 32-bit, signed binary LONG VARCHAR m + 2 characters where m = 32000 16-bit integer, count m, followed by m ASCII characters Caution: LONG VARCHAR is interpreted as VARCHAR (64000), which means that the combination of the client-side session character set and the server-side storage character set can cause a LONG VARCHAR specification in a DML USING clause to mean something other than VARCHAR(64000). Therefore, it is recommended that LONG VARCHAR be specified only if it is known that both the server-side and client-side character sets are single-byte. 100 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 28: Input Length and Field Descriptions (continued) Data Type Length Description NUMBER(p,s) maxsize = 19 The first two forms are fixed point NUMBER. Other forms are floating point NUMBER. NUMBER(p) For more information on the NUMBER data type, NUMBER see SQL Data Types and Literals (B035-1143). or NUMBER NUMBER(*,y) PERIOD(DATE) max=8 bytes 4-byte, signed integer flipped to client form. This integer represents a date in the same manner as for a DATE data type (Example: (10000*(year-1900)) + (100*month) + day). precision (n/a) The precision value specified must be between 0 and 6 inclusive. PERIOD(TIME[(n)]) max =12 bytes Second: 4-byte, signed integer flipped to client form. This integer represents the number of seconds as a scaled decimal. Hour: 1 unsigned byte. This byte represents the number of hours. Minute: 1 unsigned byte to client form. This byte represents the number of minutes precision n The precision value specified must be between 0 and 6 inclusive. PERIOD(TIME[(n)] WITH TIME ZONE) max=16 bytes Second: 4-byte, signed integer flipped to client form. This integer represents the number of seconds as a scaled decimal. Hour: 1 unsigned byte. This byte represents the number of hours. Minute: 1 unsigned byte. This byte represents the number of minutes. Time Zone_Hour: 1 unsigned byte. This byte represents the hours portion of the time zone displacement along with whether the displacement is + or -. Time Zone Minute: 1 unsigned byte. This byte represents the minutes portion of the time zone displacement. precision n The precision value specified must be between 0 and 6 inclusive. Teradata FastLoad Reference 101 Chapter 3: Teradata FastLoad Commands DEFINE Table 28: Input Length and Field Descriptions (continued) Data Type Length Description PERIOD(TIMESTAMP[(n)]) max=20 bytes Second: 4-byte, signed integer flipped to client form. This integer represents the number of seconds as a scaled decimal. Year: 2-byte, signed short integer flipped to client form. This byte represents the year value. Month: 1 unsigned byte. This byte represents the month value. Day: 1 unsigned byte. This byte represents the day of the month. Hour: 1 unsigned byte. This byte represents the number of hours. Minute: 1 unsigned byte. This byte represents the number of minutes. precision n The precision value specified must be between 0 and 6 inclusive. PERIOD(TIMESTAMP[(n)] WITH TIME ZONE) max=24 bytes Second: 4-byte, signed integer flipped to client form. This integer represents the number of seconds as a scaled decimal. Year: 2-byte, signed short integer flipped to client form. This byte represents the year value. Month: 1 unsigned byte. This byte represents the month value. Day: 1 unsigned byte. This byte represents the day of the month. Hour: 1 unsigned byte. This byte represents the number of hours. Minute: 1 unsigned byte. This byte represents the number of minutes. Time Zone_Hour: 1 unsigned byte. This byte represents the time zone displacement in hours along with whether the displacement is + or -. Time Zone Minute: 1 unsigned byte. This byte represents the time zone displacement in minutes. precision n The precision value specified must be between 0 and 6 inclusive. 102 SMALLINT 2 bytes 16-bit, signed binary VARCHAR(n) m + 2 bytes where m = 32000 16-bit integer, count m, followed by m ASCII characters VARBYTE(n) m + 2 bytes where m <= 16-bit integer, count m, followed by m bytes of data n Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 28: Input Length and Field Descriptions (continued) Data Type Length Description GRAPHIC(n) (n*2) bytes, if n is specified; otherwise, 2 bytes, as n = 1 is assumed n double-byte characters (1n is the length of the input stream in terms of double-byte characters) VARGRAPHIC(n) m + 2 bytes where m/2 <= n 2-byte integer followed by m/2 double-byte characters • Note: For both VARGRAPHIC and LONG VARGRAPHIC, m, a value occupying the first two bytes of the input data, is the length of the input in bytes, not characters. Each multibyte character-set character is 2 bytes. GRAPHIC Data Types GRAPHIC data types define multibyte character set data. Teradata FastLoad accepts GRAPHIC data in its input data when a site is defined for kanji. The DEFINE command supports these types of input data: • GRAPHIC • VARGRAPHIC • LONG VARGRAPHIC The format to accommodate multibyte character sets and data containing multibyte characters is: • “G” and “XG” for mainframe-attached systems • “XG” and the standard character string format for network-attached systems where Syntax Element Description “G” G’<....>’ where “<” and “>” represent the Shift-Out (0x0E) and Shift-In (0x0F) characters, respectively • All characters in between must be valid characters for the character set • The number of characters within the Shift-Out/Shift-In must be an even number. Teradata FastLoad Reference 103 Chapter 3: Teradata FastLoad Commands DEFINE Syntax Element Description “XG” ‘hhhh’XG where • “hh” represents a pair of hexadecimal digits (0-9 and A-F) • Each pair of hexadecimal digits represents a single GRAPHIC character. • Since a maximum of 80 bytes may be specified in a NULLIF clause, this translates to 80 pairs of hexadecimal digits. • NULLIF Data Type Restrictions and Limitations The NULLIF option nulls a column in a table when the data field is a certain value. A field with a value of zero, for example, could represent a null date. To meet this requirement, enter the field definition in the DEFINE command as: DueDate (DATE, NULLIF = 0) Teradata FastLoad compares the value entered in the NULLIF clause with the actual data row. If they match, the utility sets the appropriate indicator bit to ON for the column in that row and sends both the row and the indicator bit string to the Teradata Database. The Teradata Database then inserts a null value into the column. VARCHAR fields are checked to ascertain if the length of the NULLIF string matches the 2-byte length indicator field (in the data row). The values are compared only if they are equal. If a value is a NULLIF value that equals 10 bytes, Teradata FastLoad compares it with the first 10 bytes of the corresponding field in the data row. FastLoad does not support NULLIF clause on period data type columns. The following minimum and maximum values may not apply in some applications because of the runtime environment of the individual platform: Table 29 lists the limitations by data type. Table 29: Limitations by Data Type Data Type Limitations Examples BYTE Up to 80 hexadecimal digits, enclosed by single quotes and must be an even number. “XB” is required after the hex string. Valid examples: The total number of bytes must not exceed two times the number of bytes specified in the data description. Invalid examples: DEFINE T1(BYTE (7), NULLIF = ’01’XB); DEFINE T1(BYTE (7), NULLIF = ’0123456789ABCD’XB); DEFINE T1(BYTE (7), NULLIF = ’0’XB) ; DEFINE T1(BYTE (7), NULLIF = ’0M’XB) ; Characters must be within the range of 0-9 or A-F. 104 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 29: Limitations by Data Type (continued) Data Type Limitations Examples BYTEINT Must be within the range of -128 to 127. Valid examples: DEFINE T1(BYTEINT, NULLIF = 123) ; DEFINE T1(BYTEINT, NULLIF = -123) ; Invalid examples: DEFINE T1(BYTEINT, NULLIF = 129) ; DEFINE T1(BYTEINT, NULLIF = -129) ; CHAR, CHARS, and CHARACTERS For normal string format, from 1 to 80 bytes enclosed in single quotes. For “XC” format, up to 80 pairs of hexadecimal digits enclosed in single quotes. This must be an even number and the “XC” is required. Each pair of hexadecimal digits corresponds to a single character. Valid examples: DEFINE T1(CHAR (7), NULLIF = ’ ’) ; DEFINE T1(CHAR (7), NULLIF = ’ABCDEFG’) ; Invalid example: DEFINE T1(CHAR (7), NULLIF = ’ABCDEFGH’) ; DEFINE T1(CHAR (7), NULLIF = ’ABCDEFGH’XC) ; The total number of characters defined in the NULLIF option must not exceed the number of characters specified by the data definition. Character compare operations are case sensitive, and apply only to the first 80 bytes. DATE INTEGER format only and cannot be negative. Valid examples: DEFINE T1(INTEGER, NULLIF = 123) ; DEFINE T1(DATE, NULLIF = 941015) ; Invalid example: DEFINE T1(DATE, NULLIF=94-10-15); DECIMAL Must be specified by a zoned number less than or equal to 38 digits. The number of digits specified in the NULLIF option must not exceed the number of digits entered in the data definition. If the NULLIF value contains more digits after the decimal point than are defined, results are undefined. Valid examples: DEFINE T1(DECIMAL (5,2), NULLIF = 0) ; DEFINE T1(DECIMAL (5,2), NULLIF = 123.45) ; Invalid example: DEFINE T1(DECIMAL (6,0), NULLIF = 1234567) ; Using NULLIF for DECIMAL (other than zero) may jeopardize a NULLIF match. Teradata FastLoad Reference 105 Chapter 3: Teradata FastLoad Commands DEFINE Table 29: Limitations by Data Type (continued) Data Type Limitations Examples FLOAT Floating point values are represented differently on the Teradata Database and on some of the other platforms. Consequently, compare operations with exported floating point numbers may not function properly. A valid example: DEFINE T1(FLOAT, NULLIF = 123) ; The format for floating point numbers is: xxx.xxx or xx.xxE(+/-)yy or xE(+/-)yy or xxx The range of valid floating point values on the various platforms is: • z/OS: 1E-36 to 1E+35 • UNIX OS: 4.94065645841246544e-324 to 1.79769313486231470e+308 Windows: 3.4e-38 to 3.4e+38 GRAPHIC and VARGRAPHIC From 1 to 80 characters and must be enclosed in single quotes. For mainframe-attached systems, the quoted string must be preceded by “G” or followed by “XG”. For network-attached systems, the quoted string may be followed by “XG”, but cannot be preceded by “G”. Valid examples on mainframe-attached systems: DEFINE T1(GRAPHIC(4), NULLIF = G'<ABCDEFGH>'); DEFINE T1(GRAPHIC(4), NULLIF = G'<01234567>'); Invalid examples on mainframe-attached systems: DEFINE T1(GRAPHIC(4), NULLIF = G'<01234567ABCD>'); DEFINE T1(GRAPHIC(4), NULLIF = G'ABCD0123'); When using the "G" format, the total number of characters defined by the NULLIF clause must not exceed two times the number of bytes specified in the data description. Valid examples on network-attached systems: When using the "XG" format, the total number of hexadecimal digits defined by the NULLIF clause must not exceed four times the number of bytes specified in the data description. Invalid examples on network-attached systems: The GRAPHIC or VARGRAPHIC string has this form: DEFINE T1(GRAPHIC(4), NULLIF = 'ABCDEFGH'); DEFINE T1(GRAPHIC(4), NULLIF = '01234567'); DEFINE T1(GRAPHIC(4), NULLIF = G'<ABCDEFGH>'); DEFINE T1(GRAPHIC(4), NULLIF = G'<01234567>'); G’<ABC>’ where <ABC> is the quoted string of valid MBC and the characters < and > represent 0x0E and 0x0F. INTEGER 106 Integer fields and date fields must be within the range of -2147483648 to 2147483647. Valid examples: DEFINE T1(INTEGER, NULLIF = 123) ; DEFINE T1(DATE, NULLIF = 941015) ; Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 29: Limitations by Data Type (continued) Data Type Limitations Examples SMALLINT Small integer fields must be within the range of Valid examples: DEFINE T1(SMALLINT, NULLIF = 123) ; DEFINE T1(SMALLINT, NULLIF = -123) ; -32768 to 32767. Invalid examples: DEFINE T1(SMALLINT, NULLIF = 32768) ; DEFINE T1(SMALLINT, NULLIF = -32769) ; VARBYTE 80 hex digits, enclosed by single quotes and must be an even number. “XB” is required after the hex string. VARCHAR For normal string format, 1-80 bytes enclosed in single quotes. For “XC” format, up to 80 pairs of hexadecimal digits, enclosed in single-quotes and must be an even number. The “XC” is required, and each pair of hexadecimal digits corresponds to a single character. • Numeric Fields The MAXIMUM and MINIMUM range for all fields is the default for each machine implementation. These values generally agree with the Teradata Database except in cases where they are limited by the implementation. These values are documented by the manufacturer or can be found in the “C” language guide for that machine. • Using Table Definitions to Define Data Use either of the following commands to retrieve a list of field names from the referenced table: HELP TABLE tname ; INSERT tname.* ; Note: Do not use both of these commands together. In addition, do not use the tname.* version of an INSERT statement when using Unicode data from the following types of sessions. For more information about this precaution, see Table 36 on page 126. • A KATAKANAEBCDIC session • A session with a character set name ending with _0I • Any session with a character set that does not support multibyte characters (for example, ASCII, or EBCDIC). When this format of the INSERT statement, is used Teradata FastLoad constructs a list of field names from the table definition. During the insert operation, the utility gets the field names and their data types from the CREATE TABLE statement used to define the table and from the table definition. The following example uses an INSERT statement to get a list of field names from a table called Employee: LOGON dbc/peterson,veep ; BEGIN LOADING Employee ERRORFILES DEFINE FILE = Accounts ; INSERT Employee.*; Teradata FastLoad Reference Etable1, Etable2 ; 107 Chapter 3: Teradata FastLoad Commands DEFINE The following example uses a HELP command to get a list of field names from a table called Employee: LOGON dbc/peterson,veep ; BEGIN LOADING Employee ERRORFILES DEFINE FILE = INFILE ; HELP TABLE Employee ; INSERT INTO Employee (EmpNum, Name) Etable1, Etable2 ; VALUES (:EmpNum, :Name) ; Note: With either of these examples, a DEFINE command specifying either the input data source or INMOD parameter must also be entered. When a DEFINE command does not fit on one input line, enter either: • The command on several lines or • Several DEFINE commands In either case, Teradata FastLoad concatenates the field definitions until an INSERT statement is entered. Also, when more than one DEFINE command is entered, the field definitions in all must appear in the same order as they do in the input data record, just as they would if they were entered in a single DEFINE command. And, only a FILE or INMOD declaration can be in one of the DEFINE commands. • Using ANSI/SQL DateTime Data Types When the DATEFORM command is used to specify ANSIDATE as the DATE data type, Teradata FastLoad internally converts each DATE field to a CHAR(10) field. All ANSI/SQL DateTime TIME, TIMESTAMP, and INTERVAL data types must be converted to fixed-length CHAR data types to specify column/field names in a Teradata FastLoad DEFINE command. After the conversion to fixed-length CHAR data type, if UTF-16 session character set is used, the size should be doubled, and if UTF-8 session character set is used, the size should be tripled. For the conversion specifications and format examples for each ANSI/SQL DateTime specification, see Table 30. Table 30: ANSI/SQL DateTime Specifications DATE Convert to: CHAR(10) Format: Example: yyyy/mm/dd 1998/01/01 TIME TIME (n) Where n is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: 108 CHAR(8 + n + (1 if n > 0, otherwise 0)) Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 30: ANSI/SQL DateTime Specifications (continued) Format (n = 0): Example: hh:mm:ss 11:37:58 Format: (n = 4): Example: hh:mm:ss.ssss 11:37:58.1234 TIMESTAMP TIMESTAMP (n) Where n is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(19 + n + (1 if n > 0, otherwise 0)) Format (n = 0): Example: yyyy-mm-dd hh:mm:ss 1998-09-04 11:37:58 Format (n = 4): Example: yyyy-mm-dd hh:mm:ss.ssss 1998-09-04 11:37:58.1234 TIME WITH TIME ZONE TIME (n) WITH TIME ZONE Where n is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(14 + n + (1 if n > 0, otherwise 0)) Format (n = 0): Example: hh:mm:ss{±}hh:mm 11:37:58-08:00 Format (n = 4): Example: hh:mm:ss.ssss {±} hh:mm 11:37:58.1234-08:00 TIMESTAMP WITH TIME ZONE TIMESTAMP (n) WITH TIME ZONE Where n is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(25 + n + (1 if n > 0, otherwise 0)) Format (n = 0): Example yyyy-mm-dd hh:mm:ss{±}hh:mm 1998-09-24 11:37:58+07:00 Format (n = 4): Example: yyyy-mm-dd hh:mm:ss.ssss{±}hh:mm 1998-09-24 11:37:58.1234+07:00 INTERVAL YEAR INTERVAL YEAR (n) Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n) Format (n = 2): Example: yy 98 Format (n = 4): Example: yyyy 1998 Teradata FastLoad Reference 109 Chapter 3: Teradata FastLoad Commands DEFINE Table 30: ANSI/SQL DateTime Specifications (continued) INTERVAL YEAR TO MONTH INTERVAL YEAR (n) TO MONTH Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n + 3) Format (n = 2): Example: yy-mm 98-12 Format (n = 4): Example: yyyy-mm 1998-12 INTERVAL MONTH INTERVAL MONTH (n) Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n) Format (n = 2): Example: mm 12 Format (n = 4): Example: mmmm 0012 INTERVAL DAY INTERVAL DAY (n) Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n) Format (n = 2): Example: dd 31 Format (n = 4): Example: dddd 0031 INTERVAL DAY TO HOUR INTERVAL DAY (n) TO HOUR Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n + 3) Format (n = 2): Example: dd hh 31 12 Format (n = 4): Example: dddd hh 0031 12 INTERVAL DAY TO MINUTE INTERVAL DAY (n) TO MINUTE Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: 110 CHAR(n + 6) Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 30: ANSI/SQL DateTime Specifications (continued) Format (n = 2): Example: dd hh:mm 31 12:59 Format (n = 4): Example: dddd hh:mm 0031 12:59 INTERVAL DAY TO SECOND INTERVAL DAY (n) TO SECOND INTERVAL DAY TO SECOND (m) INTERVAL DAY (n) TO SECOND (m) Where • n is the number of digits, 1 through 4. (Default = 2.) • m is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(n + 9 + m + (1 if m > 0, 0 otherwise)) Format (n = 2, m = 0): Example: dd hh:mm:ss 31 12:59:59 Format (n = 4, m = 4): Example: dddd hh:mm:ss.ssss 0031 12:59:59:59.1234 INTERVAL HOUR INTERVAL HOUR (n) Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n) Format (n = 2): Example: hh 12 Format (n = 4): Example: hhhh 0012 INTERVAL HOUR TO MINUTE INTERVAL HOUR (n) TO MINUTE Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n + 3) Format (n = 2): Example: hh:mm 12:59 Format (n = 4): Example: hhhh:mm 0012:59 INTERVAL HOUR TO SECOND INTERVAL HOUR (n) TO SECOND INTERVAL HOUR TO SECOND (m) INTERVAL HOUR (n) TO SECOND (m) Teradata FastLoad Reference 111 Chapter 3: Teradata FastLoad Commands DEFINE Table 30: ANSI/SQL DateTime Specifications (continued) Where • n is the number of digits, 1 through 4. (Default = 2.) • m is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(n + 6 + m + (1 if m > 0, 0 otherwise)) Format (n = 2, m = 0): Example: hh:mm:ss 12:59:59 Format (n = 4, m = 4): Example: hhhh:mm:ss.ssss 0012:59:59.1234 INTERVAL MINUTE INTERVAL MINUTE (n) Where n is the number of digits, 1 through 4. (Default = 2.) Convert to: CHAR(n) Format (n = 2): Example: mm 59 Format (n = 4): Example: mmmm 0059 INTERVAL MINUTE TO SECOND INTERVAL MINUTE (n) TO SECOND INTERVAL MINUTE TO SECOND (m) INTERVAL MINUTE (n) TO SECOND (m) Where • n is the number of digits, 1 through 4. (Default = 2.) • m is the number of digits after the decimal point, 0 through 6. (Default = 6.) Convert to: CHAR(n + 3 + m + (1 if m > 0, 0 otherwise)) Format (n = 2, m = 0): Example: mm:ss 59:59 Format (n = 4, m = 4): Example: mmmm:ss.ssss 0059:59.1234 INTERVAL SECOND INTERVAL SECOND (n) INTERVAL SECOND (n,m) Where • n is the number of digits, 1 through 4. (Default = 2.) • m is the number of digits after the decimal point, 0 through 6. (Default = 6.) 112 Convert to: CHAR(n + m + (1 if m > 0, 0 otherwise)) Format (n = 2, m = 0): Example: ss 59 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE Table 30: ANSI/SQL DateTime Specifications (continued) Format (n = 4, m = 4): Example: • ssss.ssss 0059.1234 Using Period Data Types A period is an anchored duration. It represents a set of contiguous time granules within that duration. A period is implemented using a Period data type. A period has two elements BEGIN (the beginning element) and END (the ending element) which have an element type that is one of the three DateTime data types. For the CHAR data type, “n” represents the size of the field. For PERIOD data types, this is not the case. “n” represents the precision (number of digits in the fractional part of seconds). PERIOD data is always represented externally in binary format • Using ARRAY Data Types A column that is defined as an ARRAY data type in a Teradata table must be specified as a VARCHAR data type in the FIELD command. The external representation for an ARRAY data type is VARCHAR. The following is a sample Teradata table definition that includes a one-dimensional ARRAY data type for the COL003 column: CREATE SET TABLE SOURCE_TABLE ,NO FALLBACK , NO BEFORE JOURNAL, NO AFTER JOURNAL, CHECKSUM = DEFAULT, DEFAULT MERGEBLOCKRATIO ( EMP_ID INTEGER, EMP_NO BYTEINT, COL003 SYSUDTLIB.PHONENUMBERS_ARY, COL004 SYSUDTLIB.DECIMAL_ARY, COL005 SYSUDTLIB.INTEGER_ARY) UNIQUE PRIMARY INDEX ( EMP_ID ); The following is a sample definition for the PHONENUMBERS_ARY data type: CREATE TYPE PHONENUMBERS_ARY AS CHAR(10) CHARACTER SET LATIN ARRAY [2]; The following is a sample definition for the DECIMAL_ARY data type: CREATE TYPE DECIMAL_ARY AS DECIMAL(5,2) ARRAY[2]; The following is a sample definition for the INTEGER_ARY data type: CREATE TYPE INTEGER_ARY AS INTEGER ARRAY[2]; The following is a sample FastLoad field definitions specified with the DEFINE command for SOURCE_TABLE table: EMP_ID (INTEGER), EMP_NO (BYTEINT), COL003 (VARCHAR(47)), COL004 (VARCHAR(17)), COL005 (VARCHAR(25)) Teradata FastLoad Reference 113 Chapter 3: Teradata FastLoad Commands DEFINE In the above example, the COL003 column is defined as VARCHAR(47) because it's the maximum representation for the COL003 column in the table. The following is the calculation for the maximum representation for the COL003 column: 1 byte for the left parenthesis + 1 byte for the single quote + 10 to 20 bytes for the first element + 1 byte for the single quote + 1 byte for the comma + 1 byte for the single quote + 10 to 20 bytes for the second element + 1 byte for the single quote + 1 byte for the right parenthesis ---47 bytes The following is two samples of data for the COL003 column: Sample data 1: ('3105551234','3105551234') Sample data 2: ('''''''''''''''''''''','''''''''''''''''''''') Sample data 1 contains 2 elements of phone numbers. Sample data 2 contains 2 elements of all single quote characters. In the above example, the COL004 column is defined as VARCHAR(17) because it's the maximum representation for the COL004 column in the table. The following is the calculation for the maximum representation for the COL004 column: 1 byte for the left parenthesis + 1 to 7 bytes for the first element + 1 byte for the comma + 1 to 7 bytes for the second element + 1 byte for the right parenthesis ---17 bytes The following is two samples of data for the COL004 column: Sample data 1: (-123.45,888.10) Sample data 2: (+123.45,-888.10) In the above example, the COL005 column is defined as VARCHAR(25), because it's the maximum representation for the COL005 column in the table. The following is the calculation for the maximum representation for the COL005 column: 1 byte for the left parenthesis + 1 to 11 bytes for the first element + 1 byte for the comma + 1 to 11 bytes for the first element 114 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands DEFINE + 1 byte for the right parenthesis ---25 bytes The following is two samples of data for the COL005 column: Sample data 1: (-2147483648,+2147483647) Sample data 2: (0,0) Use the Teradata SQL "HELP TYPE" command to find out the maximum length for the ARRAY data type. For example, the information for the sample PHONENUMBERS_ARY, DECIMAL_ARY, and INTEGER_ARY ARRAY data types can look as follows: help type PHONENUMBERS_ARY; *** Help information returned. One row. *** Total elapsed time was 1 second. Name PHONENUMBERS_ARY Internal Type A1 External Type CV Max Length 47 Array(Y/N) Y Dimensions 1 Element Type CF UDT Name ? Array Scope [1:2] Total Digits ? Fractional Digits ? Contains Lob N Ordering F Ordering Category M Ordering Routine LOCAL Cast N Transform Y Method Y Char Type 1 HELP TYPE DECIMAL_ARY; *** Help information returned. One row. *** Total elapsed time was 1 second. Name DECIMAL_ARY Internal Type A1 External Type CV Max Length 17 Decimal Total Digits ? Decimal Fractional Digits Contains Lob N Ordering F Ordering Category M Ordering Routine LOCAL Cast N Transform Y Method Y Char Type 1 Array(Y/N) Y Teradata FastLoad Reference ? 115 Chapter 3: Teradata FastLoad Commands DEFINE Dimensions Element Type D UDT Name ? Array Scope [1:2] 1 HELP TYPE INTEGER_ARY; *** Help information returned. One row. *** Total elapsed time was 1 second. Name INTEGER_ARY Internal Type A1 External Type CV Max Length 25 Decimal Total Digits ? Decimal Fractional Digits Contains Lob N Ordering F Ordering Category M Ordering Routine LOCAL Cast N Transform Y Method Y Char Type 1 Array(Y/N) Y Dimensions 1 Element Type I UDT Name ? Array Scope [1:2] ? As indicated in the returned information from the HELP TYPE command, the maximum length for the sample PHONENUMBERS_ARY ARRAY data type is 47 bytes. The maximum length for the sample DECIMAL_ARY ARRAY data type is 17 bytes. The maximum length for the sample INTEGER_ARY ARRAY data type is 25 bytes. For more information about the external representations for the ARRAY data type, see SQL Data Types and Literals (B035-1143). • Using Session Character Set KANJISJIS_0S When the session character set used is KANJISJIS_0S, if the CHAR(n) or VARCHAR(n) of the table to be loaded is defined as UNICODE character set, the corresponding field size in the DEFINE command should be doubled, that is CHAR(n*2) or VARCHAR(n*2) respectively. If the CHAR(n) or VARCHAR(n) of the table to be loaded is defined as LATIN character set, the corresponding field size in the DEFINE command remains the same, that is CHAR(n) or VARCHAR(n) respectively. 116 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands END LOADING END LOADING Purpose The END LOADING command distributes all of the rows that were sent from the client to the Teradata Database during the loading phase to their final destination on the AMPs. Syntax ; END LOADING 2411A019 Usage Notes Table 31 describes the things to consider when using the END LOADING command. Table 31: Usage Notes for END LOADING Topic Usage Notes End Loading Phase The END LOADING command begins the end loading phase of a Teradata FastLoad job. During this phase, all rows are distributed on the AMPs and stored in the final Teradata FastLoad table. When the end loading phase completes, the Teradata Database removes the access locks that were placed on the three tables specified in the BEGIN LOADING command so users with the proper privileges can access them using Teradata SQL statements. Entering Teradata SQL Statements Many of the Teradata SQL statements from Teradata FastLoad cannot be entered because the utility supports only a subset of the Teradata SQL language. So, when END LOADING has completed and access to data stored in the Teradata FastLoad table or the error tables is required, it cannot be done from Teradata FastLoad. You must use BTEQ or a similar application program must be used to query these tables. Error Tables Teradata FastLoad automatically drops error tables that contain no rows when END LOADING finishes executing. Internal Checkpointing The Teradata Database uses internal checkpointing while processing an END LOADING command. Therefore, a job can be interrupted during the end loading phase without disturbing processing status. When the job is restarted, the Teradata Database resumes processing from where it left off. Example The following command example completes a Teradata FastLoad job: END LOADING ; Teradata FastLoad Reference 117 Chapter 3: Teradata FastLoad Commands END LOADING Completion Message The Teradata FastLoad completion message for the END LOADING command depends on whether the job includes a RECORD command. Table 32 describes the possible RECORD command Completion messages. Table 32: RECORD Completion Message Conditions Job 118 Teradata FastLoad Response Layout If the Job includes a RECORD command Total Records Read = 500 - skipped by RECORD command = 50 - sent to the RDBMS = 450 Total Error Table 1 = 25 Total Error Table 2 = 0 Total Inserts Applied = 425 Total Duplicate Rows = 0 If the Job does not include a RECORD command Total Total Total Total Total Records Read = Error Table 1 = Error Table 2 = Inserts Applied = Duplicate Rows = 500 25 0 475 0 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands ERRLIMIT ERRLIMIT Purpose The ERRLIMIT command limits the number of records that can be rejected while inserting data into the Teradata FastLoad table. Syntax ; ERRLIMIT n 2411A021 where Syntax Element Description n Maximum number of records that can be rejected before executing the END LOADING command. The default error limit value is 1000000. Usage Notes Table 33 describes the things to consider when using the ERRLIMIT command. Table 33: Usage Notes for ERRLIMIT Topic Usage Notes Limiting Insertion Errors Use the ERRLIMIT command to limit the number of insertion errors captured in the first error table (errortname1) during the loading phase of a job. Processing terminates when the number of errors encountered reaches the error limit. If, for example, errors are not expected in the input data, set the error limit value to one. In this case, the job terminates when any record causes an error. Note, however, that when the specified error limit is reached, Teradata FastLoad continues processing until each session completes its current data block. This continued processing can cause the total number of error rows captured in the first error table to exceed the ERRLIMIT specification. Restarting a Job Teradata FastLoad Reference A job can be restarted from the last checkpoint if it terminates because the error limit is reached. If checkpoints were not taken, restart the job from the beginning. 119 Chapter 3: Teradata FastLoad Commands ERRLIMIT Example The following command example terminates the job after 20 errors: ERRLIMIT 20 ; Completion Message The Teradata FastLoad completion message is: Error limit set to :20 120 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands HELP HELP Purpose The HELP command returns the syntax of all Teradata FastLoad commands and lists the Teradata SQL statements supported by Teradata FastLoad. Syntax ; HELP 2411A020 Usage Notes Table 34 describes the things to consider when using the HELP command. Table 34: Usage Notes for HELP Topic Usage Notes Using the HELP Command The HELP command is intended for online use. Return Message Symbols The HELP command return messages use these symbols: • [ ] (brackets) indicate an optional entry. • { } (curly braces) indicate a choice of entries, one of which must be selected. Example The following command example returns a list of Teradata FastLoad commands: HELP ; Completion Message The Teradata FastLoad completion message is: Listed below is the syntax of each Teradata FastLoad command. Single-line commands may be preceded by a [.] or terminated by a [;]. Multi-line commands must NOT be preceded by a [.] but must be terminated by a [;]. SQL statements must NOT be preceded by a [.] and MUST be terminated by a [;]. AXSMOD name [ "<init-string>" ] ; BEGIN LOADING [dbname.]tname1 ERRORFILES [dbname.]errortname1, [dbname.]errortname2 [ CHECKPOINT integer ] [ INDICATORS ] ; CLEAR ; { INTEGERDATE } DATEFORM { ----------- } ; Teradata FastLoad Reference 121 Chapter 3: Teradata FastLoad Commands HELP { ANSIDATE DEF[INE] [ fieldname (data [,fieldname (data [ { FILE=filename [ { [ { INMOD=name END LOADING ; ERRLIMIT n ; } type [,NULLIF [=] value ]) ... type [,NULLIF [=] value ])] ] } ] } ] ; } ] HELP ; HELP TABLE tname ; The INSERT statement has two formats: 1. INS[ERT] [INTO] tname.* ; 2. INS[ERT] [INTO] tname (cname [... ,cname]) VALUES (:fieldname [... ,fieldname]) ; LOGOFF ; LOG[ON] [tdpid/] username,password [ , 'acctid' ] ; { OFF } } [ EXIT [name] [TEXT "string"] ] { --NOTIFY { LOW } [ MSG [text] ] ; { MEDIUM } [ QUEUE [options] ] { HIGH } OS oscommand ; QUIT ; RECORD [startrecordnumber] [THRU endrecordnumber] ; SESSIONS n|* [ m|* ] ; { FORMATTED } SET RECORD { --------} ; { UNFORMATTED } { VARTEXT [ c ] [DISPLAY_ERRORS] [NOSTOP] } { 0 - 255 } { ASCII } } ; SET SESSION CHARSET { ----{ KanjiEUC_0U } { KanjiSJIS_0S } SHOW ; SHOW VERSION[S] ; SLEEP n ; TENACITY n ; The following DBS/SQL statements are supported by the FastLoad utility: CREATE TABLE DATABASE dbname ; DEL[ETE] FROM tname [ ALL ] ; DROP TABLE tname ; Note: Replace “FILE=” with “DDNAME=” for z/OS. 122 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands HELP TABLE HELP TABLE Purpose The HELP TABLE command creates a list of field names by querying the Teradata Database and deriving the DEFINE list from the table definition. Syntax tname HELP TABLE ; dbname. 2411A023 where Syntax Element Description tname Table to be queried. dbname Database in which the table resides. Usage Notes Table 35 describes the things to consider when using the HELP TABLE command. Table 35: Usage Notes for HELP TABLE Topic Usage Notes Using DEFINE Commands If the HELP TABLE command is used to define field names, a DEFINE command must still be used to specify the input data source name or INMOD routine. UDT column If the table contains a UDT column, an external representation of the UDT is returned. For example, if the user defines a USDollar data type as Decimal(13,2) and defines a column as USDollar type in the table, Decimal(13,2) is returned as the data type of this column. Teradata FastLoad Reference 123 Chapter 3: Teradata FastLoad Commands HELP TABLE Table 35: Usage Notes for HELP TABLE (continued) Topic Usage Notes Using a CLEAR Command When using two HELP TABLE commands in the same Teradata FastLoad job, using a CLEAR command before the second one cancels the first. The following command example produces a list of only the Department table: HELP TABLE Employee ; CLEAR ; HELP TABLE Department ; Entering the two HELP TABLE commands without a CLEAR command, as in the following example, produces a list of both the Employee and Department tables: HELP TABLE Employee ; HELP TABLE Department ; Unicode Session Character Set Limitation Teradata FastLoad uses the number of bytes of storage returned from Teradata Database to construct the USING clause of a load operation. Therefore, when the session character set is UTF-8 or UTF-16, the MAX LENGTH returned from the database is not the actual byte count for the Unicode column, meaning that the internally generated USING clause does not properly reflect the structure of the input data stream. Instead of using HELP TABLE to describe the structure of input data, use the DEFINE command when the session character set is UTF-8 or UTF-16. Example The following command example builds a list of field names from the table definition for a table named Employee: HELP TABLE Employee ; The SHOW command can be used to verify the field names defined in the HELP TABLE command: SHOW ; 124 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands INSERT INSERT Purpose INSERT is a Teradata SQL statement that inserts data records into the rows of the Teradata FastLoad table. Note: FastLoad also supports temporal syntaxes like CURRENT VALIDTIME, SEQUENCED VALIDTIME, VALIDTIME, NONSEQUENCED VALIDTIME and NONTEMPORAL clauses prefixed in INSERT/INS statement. Syntax ; tname INSERT . INTO INS * , values ( cname ) values VALUES , ( :fieldname ) 2411A024 where Syntax Element Description tname Name of the table into which rows are inserted. cname Name of column to receive a new row value during the insert operation. For each cname defined, a corresponding fieldname must be specified. A list of column names can be defined in any order. The column names do not have to be defined in the same order as they appear in the CREATE TABLE statement. fieldname Field name that was defined in a previous DEFINE command. During the insert operation, Teradata FastLoad inserts the field in the input data record that was assigned to the fieldname into the corresponding column (cname) of the Teradata FastLoad table. .* Wildcard specification that all columns in tname that are used to construct the list of field names be used in the insert operation. When the wildcard specification is used, Teradata FastLoad queries the Teradata Database for all of the column names and uses them to construct a valid INSERT statement. Usage Notes The following table describes the things to consider when using the INSERT statement. Teradata FastLoad Reference 125 Chapter 3: Teradata FastLoad Commands INSERT Table 36 describes the things to consider when using the INSERT command. Table 36: Usage Notes for INSERT Topic Usage Notes Required Privileges To use the INSERT statement, the user ID associated with the Teradata FastLoad job must have INSERT privilege on the specified table. Inserting Field Values During the insert operation, field values are inserted in the table in the order in which the columns are listed in the CREATE TABLE statement. If field values in the input data are stored in the same order as columns are defined in the CREATE TABLE statement for the Teradata FastLoad table, a list of column names does not need to be specified in the INSERT statement (for instance, INSERT INTO table1 VALUES (:f1, :f2). When the second format of the INSERT statement is used, a list of field names is constructed from the definition of the table. During the insert operation, field names and their data types are taken from the CREATE TABLE statement and used to define the table. The field name definitions are established in the order in which columns are defined in the CREATE TABLE statement. So, the fields in each data record must be in the same order as the columns in the definition of the table. Using DEFINE Commands When using the second form of the INSERT statement, use the DEFINE command to specify the name of the input data source or INMOD routine used in the Teradata FastLoad job. If a DEFINE command that defines one or more fields before the INSERT statement is entered, Teradata FastLoad appends the field definitions to the definitions constructed from the INSERT statement. Note: The colon character preceding the input field name descriptions (:fieldname) indicates that a corresponding DEFINE field must exist. If the INSERT statement does not include: fieldname expressions, then Teradata FastLoad transmits the command to the Teradata Database intact, without linking it with a previous DEFINE command. 126 Using SHOW TABLE Statements If a Teradata SQL SHOW TABLE statement is used to display the exact definition of a table, you must do so from BTEQ or another application. Teradata FastLoad does not support this statement. ANSI/SQL DateTime Specifications The ANSI/SQL DATE, TIME, TIMESTAMP, and INTERVAL DateTime data types in Teradata SQL CREATE TABLE statements can be used. They can be specified as column/field modifiers in INSERT statements. They must be converted to fixed-length CHAR data types when specifying the column/field names in the Teradata FastLoad DEFINE command. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands INSERT Table 36: Usage Notes for INSERT (continued) Topic Usage Notes Using Unicode Data Caution: Do not use the tname.* version of an INSERT statement when using Unicode data from any of the following: • A KATAKANAEBCDIC session • A session with a character set name ending with _0I • Any session with a character set that does not support multibyte characters (for example, ASCII or EBCDIC) In addition to the field names from the referenced tables, these functions return byte/character counts that Teradata FastLoad uses internally to construct the USING clause for the subsequent load operation. Because of the byte and character count conversions that take place when importing and exporting CHAR and VARCHAR data between a client system and the Teradata Database, the internally generated USING clause does not properly reflect the structure of the input data stream. Unicode Session Character Set Limitation For information, see Table 35 on page 123. Example 1 The following command example defines the EmpRecs input data source and the fields in each record (Emp_Number and Emp_Name): DEFINE Emp_Number (INTEGER), Emp_Name (VARCHAR(30)), FILE=EmpRecs ; INSERT INTO Employee (EmpNo, Name) VALUES (:Emp_Number, :Emp_Name) ; The INSERT statement defines the table and columns to receive new data values. For each data record, the value in the first field (Emp_Number) is inserted in the EmpNum column and the value in the second field (Emp_Name) is inserted in the Name column. Example 2 The following command example establishes a list of field names from the definition of the Teradata FastLoad table: DEFINE FILE=InFile; INSERT OldTable.* ; The DEFINE command specifies the input data source (InFile) and defines each field to be inserted in the Teradata FastLoad table (OldTable). Example 3 Sometimes the records in an input data source contain data that does not belong in the Teradata FastLoad table. If, for example, each record contains 100 bytes of extra data, a dummy field can be defined in the DEFINE command that is not referenced in the INSERT statement. Teradata FastLoad Reference 127 Chapter 3: Teradata FastLoad Commands INSERT The following command example constructs a list of field names that match the current definition of NewTable appended to the definition of the extra data item: DEFINE ExtraData (CHAR (100)) FILE = InFile; INSERT NewTable.*; When extra data is used in the input data source with the INSERT TABLE .* feature, the extra data must be located at the beginning of each record. This feature cannot be used if the extra data occurs in the middle or at the end of the records. In this case, explicitly define each data item in the data source and each item in the values clause of the INSERT statement. 128 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands LOGDATA LOGDATA Purpose Supplies parameters to the LOGMECH command beyond those needed by the logon mechanism, such as user ID and password, to successfully authenticate the user. The LOGDATA command is optional. Whether or not parameters are supplied and the values and types of parameters depend on the selected logon method. LOGDATA is only available on network-based platforms. Syntax .LOGDATA logdata_string ; 2411A036 where Syntax Element Description logdata_string Parameters for the logon mechanism specified using “LOGMECH” on page 130 For information about the logon parameters for supported mechanisms, see Security Administration (B035-1100). The string is limited to 64 KB and must be in the session character set. Note: Make sure that the string ends with a semicolon. Usage Notes For more information about logon security, see Security Administration (B035-1100). Example If used, the LOGDATA and LOGMECH commands must precede the LOGON command. The commands themselves may occur in any order. The following example demonstrates using the LOGDATA, LOGMECH, and LOGON commands in combination to specify the Kerberos logon authentication method and associated parameters: .logmech KRB5; .logdata joe@domain1@@mypassword; .logon cs4400s3; Teradata FastLoad Reference 129 Chapter 3: Teradata FastLoad Commands LOGMECH LOGMECH Purpose Identifies the appropriate logon mechanism by name. If the mechanism specified requires parameters other than user ID and password for authentication, the LOGDATA command provides these parameters. The LOGMECH command is optional and available only on network-attached systems. Syntax .LOGMECH logmech_name ; 2409A053 where Syntax Element Description logmech_name Defines the logon mechanism For a discussion of supported logon mechanisms, see Security Administration (B035-1100). The name is limited to 8 bytes; it is not case- sensitive. Usage Notes Every session to be connected requires a mechanism name. If none is supplied, a default mechanism can be used instead, as defined on either the server or client system in an XML-based configuration file. For more information about logon security, see Security Administration (B035-1100). Example If used, the LOGDATA and LOGMECH commands must precede the LOGON command. The commands themselves may occur in any order. The following example demonstrates using the LOGDATA, LOGMECH, and LOGON commands in combination to specify the Windows logon authentication method and associated parameters: .logmech NTLM; .logdata joe@domain1@@mypassword; .logon cs4400s3; 130 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands LOGOFF LOGOFF Purpose The LOGOFF command ends Teradata FastLoad sessions and exits from the Teradata Database. The LOGOFF and QUIT commands may be used interchangeably. Syntax ; LOGOFF QUIT 2411A013 Usage Notes Table 37 describes the things to consider when using the LOGOFF command. Table 37: LOGOFF Command Usage Notes Topic Usage Notes Pausing Teradata FastLoad If the LOGOFF command is entered after a BEGIN LOADING command, but before the END LOADING command, the Teradata FastLoad job pauses and can be restarted later. Locked Tables When a Teradata FastLoad job pauses during the loading phase, the Teradata Database locks the tables named in the BEGIN LOADING command. The tables remain locked until an END LOADING command is entered. Termination Return Codes When a Teradata FastLoad job terminates, the utility returns a code indicating the way the job completed: • Code 0—Normal completion. The job completed successfully and according to the specified plan. • Code 4—Warning. A warning condition occurred; for example, a job deviated from normal or from the specified plan, but still completed successfully. A warning may indicate deviation from the plan; for example, the number of sessions specified were not actually used, or a part of the job did not run. Warning conditions do not terminate the job. • Code 8—User error. A user error, such as a syntax error in the job script, terminated the job. • Code 12—Severe error. A fatal error terminated the job. A fatal error is any error other than a user error. Teradata FastLoad Reference 131 Chapter 3: Teradata FastLoad Commands LOGOFF Example The following command example ends Teradata FastLoad and exits the Teradata Database: LOGOFF ; Completion Message The completion message indicates that the Teradata FastLoad job was either paused or terminated, depending on whether the job was in the loading phase or not: Table 38 describes the possible LOGOFF command Completion messages. Table 38: LOGOFF Command Completion Messages 132 Job Teradata FastLoad Response Layout If the Job is in the loading phase **** 20:36:06 Logging off all sessions **** 20:36:11 Total processor time used = '0.499203 Seconds' . Start : Thu Sep 20 20:35:22 2012 . End : Thu Sep 20 20:36:11 2012 . Highest return code encountered = '4'. **** 14:19:36 FDL4818 FastLoad Paused If the Job is not in the loading phase **** 20:36:06 **** 20:36:11 Seconds' . . . **** 20:36:11 Logging off all sessions Total processor time used = '0.499203 Start : End : Highest FDL4818 Thu Sep 20 20:35:22 2012 Thu Sep 20 20:36:11 2012 return code encountered = '0'. FastLoad Terminated Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands LOGON LOGON Purpose The LOGON command establishes one or more Teradata FastLoad sessions with the Teradata Database. Syntax Standard LOGON Syntax ; username, pasword LOGON tdpid/ ,'acctid' 2411A022 Single Sign-on LOGON Syntax ; LOGON tdpid/ username , password ,'acctid' 2411A006 where Syntax Element Description username If the database does not support EON feature, a user identifier can have up to 30 bytes. If the database supports EON feature, a user identifier can have up to 128 characters. password Password associated with the username. If the database does not support EON feature, a password can have up to 30 bytes. If the database supports EON feature, a password can have up to 128 characters. Passwords that contain special characters must be enclosed in double quotes. Teradata FastLoad Reference 133 Chapter 3: Teradata FastLoad Commands LOGON Syntax Element Description tdpid Optional character string that identifies the name of a TDP. The tdpid string can have from 1 to 8 characters. Network-attached systems: The tdpid string can have up to 256 characters and can be a domain name system (DNS) name. The tdpid is the name of the host entered in the network hosts file. If this field is not supplied, tdpid defaults to the TDP established for the user by the system administrator. Mainframe-attached systems: The tdpid string must be in the form: TDPn where n is the TDP identifier. acctid Account associated with the username If the database does not support EON feature, a account identifier can have up to 30 bytes. If the database supports EON feature, a account identifier can have up to 128 characters. If omitted, Teradata FastLoad uses the default account identifier defined when the user was created. Usage Notes Table 39 describes the things to consider when using the LOGON command. 134 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands LOGON Table 39: Usage Notes for LOGON Topic Usage Notes Logon Parameters For standard logon, the parameters (tdpid, username, password, and acctid) are used in all sessions established with the Teradata Database. The LOGON command may occur only once. For single sign-on logon, if the Gateway to Teradata Database is configured to use single sign-on (SSO), and the user is already logged on to the Teradata client machine, the machine name, user name, and password are not required in the LOGON command. The user name and password combination specified when the user logged on to the Teradata client machine are authenticated via network security for a single sign-on such that valid Teradata users will be permitted to log on to the Teradata Database. The use of SSO is strictly optional, unless the Gateway has been configured to accept only SSO-style logons. To connect to a Teradata Database other than the one currently logged on, the TDPid must be included in the LOGON command. If the TDPid is not specified, the default contained in clispb.dat will be used. Refer to the Teradata Call-Level Interface Version 2 Reference for Network-Attached Systems (B035-2418) for information about setting defaults. To be interpreted correctly, the TDPid must be followed by the slash separator (‘/’) to distinguish the TDPid from a Teradata Database username. For example, to connect to slugger, enter one of the following: .LOGON slugger/; .LOGON slugger/,,'acctinfo'; If an account ID is to be used, the optional account ID must be specified in the LOGON command. Note: To prevent the password from appearing in the script, use Teradata Wallet. Refer to Security Administration (B035-1100) and the appropriate installation guide for more information. Starting Teradata FastLoad The LOGON command starts a Teradata FastLoad job and automatically establishes: • Two Teradata SQL sessions • A number of Teradata FastLoad sessions Note: When Teradata FastLoad attempts to connect the Main SQL session the first time and the Teradata Database is down, Teradata FastLoad displays an error message and terminates. Note: When Teradata FastLoad attempts to connect the Main SQL session but not the first time, or the Auxiliary SQL session, or the data sessions and the Teradata Database is down. Teradata FastLoad retries to connect 16 times; if the Teradata Database is still down, Teradata FastLoad displays an error message and terminates. Number of Sessions Teradata FastLoad Reference The number of Teradata FastLoad sessions depends on the system limitations described in “Session Limits” on page 62. 135 Chapter 3: Teradata FastLoad Commands LOGON Table 39: Usage Notes for LOGON (continued) Topic Usage Notes Number of Sessions (continued) Teradata FastLoad attempts to connect sessions, in groups of 16, until either: Reported Sessions After a LOGON command is entered, Teradata FastLoad reports the number of Teradata FastLoad sessions that are logged on. The Teradata SQL sessions are not included in this report. Character Support Current multi-byte character support of an object name is limited to 30 bytes for the database that does not support EON and limited to 128 characters for the database that supports EON. • The number of sessions specified with a SESSIONS command are connected • A Teradata Database Error 2632 is returned • Some other session limit is reached Note: If the number that was specified in a SESSIONS command exceeds the number of available sessions, Teradata FastLoad logs only the number of available sessions. The limitation applies to username, passwd, and account in the logon string. CLI treats logon strings of UTF-8 and other Teradata-supported multi-byte character sets (Chinese, Japanese, Korean) as ASCII. CLI does not convert before parsing, so some logon strings with multi-byte characters (or a combination of multi-byte characters plus ASCII) could fail if the total size of an object name is over 30 bytes for the database that does not support EON and is over 128 characters for the database that supports EON. The work around is to create object names that are less than 30 bytesfor the database that does not support EON are less than 128 characters for the database that supports EON. Example The following command example logs on user Peterson: LOGON DBC/peterson,HTims ; Completion Message The Teradata FastLoad completion message is: Number of FastLoad sessions connected = 8 FDL4808 LOGON successful 136 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands NOTIFY NOTIFY Purpose The NOTIFY command specifies a user exit or predefined action to be performed whenever certain significant events occur during a Teradata FastLoad job. (For a listing of events that cause notifications, see Table 40 on page 139.) The notify function is especially useful in operator-free environments where job scheduling relies heavily on automation to optimize system performance. For example, by writing an exit in C (without using CLIv2) and using the NOTIFY… EXIT option, a routine to detect whether a Teradata FastLoad job succeeds or fails, how many records were loaded, what the return code was for a failed job, and so on can be provided. Note: The Teradata FastLoad NOTIFY command applies only to the job that immediately follows it. Syntax NOTIFY ; OFF LOW EXIT EXITEON name TEXT "string " MSG "string " QUEUE option MEDIUM HIGH EXIT EXIT64 EXITEON MSG name TEXT "string " "string " 2414A005 where Syntax Element Description OFF Default. No notification of events is to be provided. LOW Notification is to be provided for those events signified by “Yes” in the Low Notification Level column of Table 40. MEDIUM Notification is to be provided for those events signified by “Yes” in the Medium Notification Level column of Table 40. HIGH Notification is to be provided for those events signified by “Yes” in the High Notification Level column of Table 40. EXIT A user-written exit is to be called at the appropriate time. Teradata FastLoad Reference 137 Chapter 3: Teradata FastLoad Commands NOTIFY Syntax Element Description name The name of a user-supplied library with an entry point named _dynamn. The default library names are: • libnotfyext.dll for Windows • libnotfyext.so for UNIX platforms • NOTFYEXT for z/OS platforms. TEXT "string" A user-supplied string of up to 80 characters that Teradata FastLoad passes to the named exit routine The string specification must be enclosed in double-quote characters ("). MSG "string" A user-supplied string of up to 16 characters that Teradata FastLoad logs on to: • The operator console on mainframe-attached z/OS client systems • The system log or EventLog service on network-attached UNIX or Windows systems The string specification must be enclosed in double quote characters. This service is not available on Windows 98. QUEUE Specifies that a queue is to be manipulated via ENQ or DEQ See Table 41 for more details. This option is valid only for z/OS. option One of the following: RNAME defines a quoted string of up to 255 characters. The default is TRDUSER. SCOPE defines one of the following: • JOB specifies that the QUEUE is local to the job (including all of the job steps). This is the default. • SYSTEMS specifies that the QUEUE is global to all computers in the complex. • SYSTEM specifies that the QUEUE is global to the computer on which it is running. NOBLOCK specifies that if the ENQ blocks for any reason, it must return an error instead. This is a fatal error for the job. The default, an implied BLOCK (there is no BLOCK keyword), means that the ENQ will wait for the QUEUE. EXIT64 A user-written exit is to be called at the appropriate time. EXITEON A user-written exit is to be called at the appropriate time. Table 40 lists events that create notification. 138 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands NOTIFY Table 40: Events that Create Notifications Notification Level Event Low Medium High Signifies Initialize Yes Yes Yes Successful processing of the NOTIFY command File or INMOD open No No Yes Successful processing of the DEFINE command Phase 1 begin No Yes Yes Beginning of the insert phase, as specified by the INSERT statement Checkpoint No No Yes Checkpoint information has been written to the restart log table Phase 1 end No Yes Yes Successful processing of the CHECKPOINT LOADING END request after the end of the insert phase Phase 2 begin No Yes Yes The END LOADING command is about to be sent to the Teradata Database Phase 2 end No Yes Yes Successful processing of the END LOADING command Error table 1 No No Yes Successful processing of the SEL COUNT(*) request for the first error table Error table 2 No No Yes Successful processing of the SEL COUNT(*) request for the second error table Teradata Database Restart No Yes Yes A crash error from the Teradata Database or the CLIv2 CLIv2 error Yes Yes Yes A CLIv2 error Teradata Database error Yes Yes Yes A Teradata Database error that will terminate Teradata FastLoad Exit Yes Yes Teradata FastLoad is terminating Yes Usage Notes Table 41 describes the things to consider when using the NOTIFY command. Table 41: Usage Notes for NOTIFY Topic Usage Notes QUEUE Notes When QUEUE with LOW option is specified, the following takes place: 1 When NOTIFY is processed, it performs an ENQ upon a QUEUE with RNAME of ‘TRDUSER’ and a scope of ‘JOB’. This call blocks until it acquires the QUEUE. 2 After the job gets the QUEUE, it continues until it reaches a specific point (such as the request completes) when it releases the QUEUE by performing a DEQ. Teradata FastLoad Reference 139 Chapter 3: Teradata FastLoad Commands NOTIFY Table 41: Usage Notes for NOTIFY (continued) Topic Usage Notes Error Handling When an error occurs, NOTIFY behaves as follows: • When NOTIFY is processed, the subsystems used by Teradata FastLoad are initialized and, if necessary, any user exits are loaded and a call is made to initialize the system log (or an ENQ is performed). • If initialization fails, FastLoad displays an error message and terminates execution with return code 12. • If anything fails after initialization, the request fails. If a user exit returns anything other than 0, a FastLoad displays an error message and terminates execution with return code 12. Restarts The following points pertain to restarts related to NOTIFY: • If a Teradata FastLoad job ends abnormally or unsuccessfully, it can be restarted and some NOTIFY-related activities are re-executed. This is an important issue with respect to writing user exits. • If a Teradata FastLoad job ends abnormally or unsuccessfully while it is holding a queue (using the QUEUE type parameter under z/OS), it releases the queue before exiting the job. Therefore, when the job restarts, ensure that it again acquires the queue before it continues processing. Creating Exit Routine When creating an exit routine, the following general procedures are constant across all operating systems: • The exit must be named _dynamn. • Success is indicated by the return of a 0 (long integer format). • Failure is indicated by the return of a nonzero value (long integer format). Different integers can be used to indicate different errors. • The parameter to the procedure is a pointer to a variable record structure. Note: For a definition of the variable record structure, see “Notify Exit Routine Example” on page 208. • A C prototype example for an exit procedure might be as follows (using a Teradata FastLoad example): long _dynamn(FLNotifyExitParm *P) The procedures for creating and using an exit routine are the same as for creating and using an INMOD routine, as described in “INMOD and Notify Exit Routines” on page 64 and Appendix D: “Compile, Link, and Execute INMOD and Notify Exit Routines.” Message Examples Table 42 lists example messages for Teradata FastLoad events. In each case, the message is preceded by the text string specified with the MSG option of the NOTIFY command. Table 42: NOTIFY Command Message Examples 140 Teradata FastLoad Event Message Notify processed - FastLoad notify processed. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands NOTIFY Table 42: NOTIFY Command Message Examples (continued) Teradata FastLoad Event Message Phase 1 is about to begin - FastLoad phase one starting for table [N] XXXXXXX.XXXXXXX. Each checkpoint - FastLoad checkpoint complete: NNNNNNNNNN records sent. When a data file is about to be opened - FastLoad opening file filename Phase 1 completes successfully - FastLoad phase one completes: NNNNNNNNN records sent. Phase 2 completes successfully - FastLoad phase two completes: NNNNNNNNN records sent. Teradata FastLoad Reference 141 Chapter 3: Teradata FastLoad Commands OS OS Purpose The OS command submits an operating system command to the client environment during a Teradata FastLoad session. Syntax ; OS command 2411A025 where Syntax Element Description command Any command that is valid for the client operating system Usage Notes The OS command must terminate with a semicolon. UNIX Examples Table 43 lists examples of OS command used on a UNIX client system. Table 43: OS Command Examples on a UNIX Client System Command Example ls The following command example lists the files in a directory on the UNIX operating system and then returns to Teradata FastLoad: OS ls ; exec sh The following command example accesses the UNIX shell: OS exec sh ; Then, at the client system prompt, enter other UNIX commands, such as: $ pg myfile.one $ cp oldfile newfile $ cd draft Press CTRL + D to exit the UNIX environment and return to Teradata FastLoad. Windows Examples Table 44 shows an example procedure for the OS command used on a Windows client system. 142 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands OS Table 44: Example OS Command on Windows Client System Command Example dir The following command example lists the files in a directory on the Windows operating system and then returns to Teradata FastLoad: OS dir; command The following command example accesses the Windows operating system: OS command; At the client system prompt, other commands can be entered, such as: c:\teradata\bin>type myfile.one c:\teradata\bin>edit myfile.one c:\teradata\bin>exit Enter exitto exit the Windows environment and return to Teradata FastLoad. z/OS Example Table 45 shows an example procedure for the OS command used on an z/OS client system. Teradata FastLoad Reference 143 Chapter 3: Teradata FastLoad Commands OS Table 45: Example Procedure for OS Command on z/OS Client System Command Example TSO Setup Procedure Before using the OS command on z/OS, the time-sharing option (TSO) must be set up to run Teradata FastLoad interactively, as in the following example: / ***************************************************************/ /* */ /* THIS CLIST INVOKE IBM_C VERSION OF FASTLOAD. */ /* */ /*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*/ /* */ /* PARAMETER USE */ /* ========= === */ /* DBCPFX = HIGH LEVEL QUALIFIER FOR "APPLOAD" LIBRARY. */ /* DEFAULT IS "DBC". */ /* */ / ***************************************************************/ PROC 0 DBCPFX(<APPLOAD LIBRARY>) CONTROL NOMSG PROMPT WRITE 'FASTLOAD CLIST TTU14.10' ALLOC FI(CTRANS) DA('IMB_C LinkLib') SHR ALLOC FI(SYSIN) DA(*) ALLOC FI(SYSPRINT) DA(*) ALLOC FI(SYSOUT) DA(*) ALLOC FI(SYSTERM) DA(*) CALL '&DBCPFX..APP.L(FASTLOAD)' FREE FI(CTRANS,SYSIN,SYSPRINT,SYSOUT,SYSTERM) EXIT where • <APPLOAD LIBRARY> is the fully qualified name of the load library containing Teradata FastLoad and CLIV2 components. Note: Unlike batch, TSO does not support concatenated load libraries. • <IBM C Linklib>is the fully qualified name of the IBM C link library. Note: Input data sets can be allocated and freed. TSO Command After setting up TSO to run Teradata FastLoad interactively, the Teradata FastLoad OS command can be used to enter any valid TSO command: OS <TSO command>; 144 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands QUIT QUIT Purpose The QUIT command ends Teradata FastLoad sessions and exits from the Teradata Database. The LOGOFF and QUIT commands may be used interchangeably. Syntax ; QUIT LOGOFF 2411A026 Usage Notes Table 46 describes the things to consider when using the QUIT command. For more information about restarting a paused job, see “Restart a Paused Teradata FastLoad Job” on page 49. Table 46: Usage Notes for QUIT Topic Usage Notes Pausing Teradata FastLoad If the QUIT command is entered after a BEGIN LOADING command, but before the END LOADING command, the Teradata FastLoad job pauses, and can be restarted later. Locked Tables When a Teradata FastLoad job pauses during the loading phase, the Teradata Database locks the tables named in the BEGIN LOADING command. The tables remain locked until an END LOADING command is entered. Terminating Return Codes When a Teradata FastLoad job terminates, the utility returns a code indicating the way the job completed: • Code 0—Normal completion. The job completed successfully and according to the specified plan. • Code 4—Warning. A warning condition occurred. Warning conditions do not terminate the job. • Code 8—User error. A user error, such as a syntax error in the Teradata FastLoad job script, terminated the job. • Code 12—Fatal error. Job is terminated. A fatal error is any error other than a user error. Teradata FastLoad Reference 145 Chapter 3: Teradata FastLoad Commands QUIT Example The following command example ends Teradata FastLoad and exits the Teradata Database: QUIT ; Completion Message The completion message indicates that the Teradata FastLoad job was either paused or terminated, depending on whether the job was in the loading phase or not: • If the job is in the loading phase, then Teradata FastLoad responds: **** 20:36:06 Logging off all sessions **** 20:36:11 Total processor time used = '0.499203 Seconds' . Start : Thu Sep 20 20:35:22 2012 . End : Thu Sep 20 20:36:11 2012 . Highest return code encountered = '4'. **** 20:36:11 FDL4818 FastLoad paused • If the job is not in the loading phase, then Teradata FastLoad responds: **** 20:36:06 Logging off all sessions **** 20:36:11 Total processor time used = '0.499203 Seconds' . Start : Thu Sep 20 20:35:22 2012 . End : Thu Sep 20 20:36:11 2012 . Highest return code encountered = '0'. **** 20:36:11 FDL4818 FastLoad Terminated 146 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands RECORD RECORD Purpose The RECORD command defines the records of the input data source at which Teradata FastLoad processing starts and ends. Syntax ; RECORD startrecordnumber THRU endrecordnumber 2411A027 where Syntax Element Description startrecordnumber Record at which processing begins. The default is record 1. THRU Keyword that introduces the optional endrecordnumber parameter. endrecordnumber Record after which processing ends. The endrecordnumber number must be equal to or greater than startrecordnumber. Usage Notes Table 47 describes the things to consider when using the RECORD command. Table 47: Usage Notes for RECORD Topic Usage Notes Entering the RECORD Command Enter the RECORD command before the INSERT statement in the Teradata FastLoad job. Restarting Teradata FastLoad Jobs When a job restarts, if the CHECKPOINT option is enabled, the utility begins reading at the next record immediately after the last checkpointed record. Teradata FastLoad Reference If a RECORD command is not used, Teradata FastLoad reads from the first record in the data source to the last record, unless the job is restarted. 147 Chapter 3: Teradata FastLoad Commands RECORD Table 47: Usage Notes for RECORD (continued) Topic Usage Notes Invalid Record Numbers The RECORD command cannot specify invalid record numbers, such as: • An endrecordnumber less than a startrecordnumber • A negative value If an invalid record number is specified, Teradata FastLoad returns an error message: • If the error occurs before the BEGIN LOADING command, then all Teradata FastLoad sessions are logged off and the utility is exited. • If the error occurs after BEGIN LOADING, then the job pauses (all Teradata FastLoad sessions are logged off and the tables named in BEGIN LOADING remain locked until END LOADING is executed). THRU Specification If the THRU endrecordnumber parameter is not specified, Teradata FastLoad begins to read at startrecordnumber and continues until it finds the last record in the data source. Example The following command example specifies the records in the input data source starting at record 1,000 and stopping at record 20,000: RECORD 1000 THRU 20000 ; Completion Message The Teradata FastLoad completion message is: Starting record number set to :1000 Ending record number set to :20000 Example The following command example specifies the records in the input data source starting at record number 1 and stopping at record number 50,000: RECORD THRU 50000 ; Completion Message The Teradata FastLoad completion message is: Starting record number set to :1 Ending record number set to :50000 148 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands RUN RUN Purpose The RUN command invokes a specified external source as the current source of commands and statements. Syntax .RUN [FILE] fileid ; 2411B035 where Syntax Element Description fileid Data source of the external system. The external system DD (or similar) statement specifies a file. • In z/OS, the fileid is a DDNAME. • In UNIX and Windows systems, the fileid is the path name for a file and supports size of up to 1024. FILE The keyword FILE is optional. Usage Notes Table 48 describes the things to consider when using the RUN command. Table 48: Usage Notes for Run Topic Usage Notes z/OS fileid Usage Rules If a DDNAME is specified, Teradata FastLoad reads data records from the specified source. A DDNAME must obey the same construction rules as Teradata SQL column names except that: • The “at” character (@) is allowed as an alphabetic character. • The underscore character (_) is not allowed. The DDNAME must obey the applicable rules of the external system and may reference a sequential or VSAM data set. If the DDNAME represents a data source on magnetic tape, the tape may be either labeled or nonlabeled, as supported by the operating system. Teradata FastLoad Reference 149 Chapter 3: Teradata FastLoad Commands RUN Table 48: Usage Notes for Run (continued) Topic Usage Notes Executing the RUN Command After Teradata FastLoad executes the RUN command, it reads additional commands from the specified source until a LOGOFF command or end-of-file condition is encountered, whichever occurs first. An end-of-file condition automatically causes Teradata FastLoad to resume reading its commands and DML statements from the previously active source: • SYSIN for z/OS • stdin (normal or redirected) for UNIX and Windows systems Note: SYSIN/stdin remains the active input source after Teradata FastLoad processes any user-provided invocation parameters. Nested RUN Commands 150 The source specified by a RUN command can have up to five levels of nested RUN commands. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SESSIONS SESSIONS Purpose The SESSIONS command specifies how many Teradata FastLoad sessions will be logged on when a LOGON command is entered and, optionally, the minimum number of sessions required to run the job. Syntax SESSIONS ; max * min * 2411A028 where Syntax Element Description max Maximum number of sessions to log on. The max specification must be greater than zero. The default, if the SESSIONS command is not used, is one session for each AMP. min Minimum number of sessions required for the job to continue. The min specification must be greater than zero. The default, if the SESSIONS command is not used, is 1. * Minimum and maximum number of sessions. Using the asterisk character as the max specification logs on for the maximum number of sessions—one for each AMP. Using the asterisk character as the min specification logs on for at least one session, but less than or equal to the max specification. Note: 1) Specifying SESSIONS * * has the same effect as not using the SESSIONS command at all. 2) On large to very large Teradata Database configurations, the default of one session per AMP may be inappropriate. Teradata FastLoad Reference 151 Chapter 3: Teradata FastLoad Commands SESSIONS Syntax Element Description * (continued) There is no general method to determine the optimal number of sessions, because it is dependent on several factors, including, but not limited to: • Teradata Database performance and workload • Client platform type, performance, and workload • Channel performance, for mainframe-attached systems • Network topology and performance, for network-attached systems • Volume of data to be processed by the application Using too few sessions is likely to unnecessarily limit throughput. On the other hand, using too many sessions can increase session management overhead (and also reduce the number of sessions available to any other applications) and may, in some circumstances, degrade throughput. Regardless of the size of the Teradata Database configuration, for large repetitive production applications, it will usually be appropriate to experiment with several different session configurations to determine the best trade-off between resource utilization and throughput performance. For larger Teradata Database configurations, it is appropriate to establish an installation default for the maximum number of sessions that is less than one session per AMP. This can be done either via the installation configuration file (see “Teradata FastLoad Configuration File” on page 52) or via a standard runtime parameter (see “Mainframe-Attached Runtime Parameters” on page 35). An installation default for number of sessions, if specified in the configuration file, can be overridden in individual Teradata FastLoad job scripts, when necessary. Usage Notes Table 49 describes the things to consider when using the SESSIONS command. Table 49: Usage Notes for SESSIONS Topic Usage Notes DBS Support TASM If the DBS supports TASM, the SESSIONS command has no effect since the number of sessions is determined by the DBS setup rules, Please refer to TRP 541-0007249 for DBS support TASM document. If FastLoad must connect to the exact number of sessions required by the DBS, otherwise FastLoad will displays the following message and terminates the job: The number of FastLoad connections (n1) is not the same as the number of connections returned by CHECK WORKLOAD END (n2) where n1 is the number of sessions that FastLoad can connect and n2 is the number of sessions that FastLoad must connect to required by the DBS. Entering the SESSIONS Command 152 The SESSIONS command must be entered before the LOGON command in the Teradata FastLoad job. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SESSIONS Table 49: Usage Notes for SESSIONS (continued) Topic Usage Notes Session Number Limits Regardless of the number of sessions specified, the actual number of sessions Teradata FastLoad uses is limited to the number of AMPs available on the Teradata Database. Thus, there is no guarantee that the number of sessions specified in the command will actually be logged on. Reported Number of Sessions Teradata FastLoad reports the number of sessions logged on when a LOGON command is executed. Invalid Number of Sessions The maximum relevant number of sessions which can be specified is 32767. Teradata FastLoad disregards any larger number and logs on for as many sessions as it can, one session per available AMP as indicated in the Teradata FastLoad error message: FDL4867 Invalid number of sessions requested FastLoad will log on as many sessions as possible Example 1 The following example specifies five Teradata FastLoad sessions: SESSIONS 5 ; Example 2 The following example specifies ten Teradata FastLoad sessions, with a minimum of five: SESSIONS 10 5 ; Completion Message The Teradata FastLoad completion message is: FDL4866 SESSIONS command accepted Error Message If an invalid value is specified, Teradata FastLoad responds with the following error message: FDL4867 Invalid number of sessions requested FastLoad will log on as many sessions as possible. Teradata FastLoad Reference 153 Chapter 3: Teradata FastLoad Commands SET RECORD SET RECORD Purpose The SET RECORD command specifies the format of the input data for Network-Attached platform as: • Formatted • Unformatted • Binary • Text • Variable-length text The SET RECORD command specifies the format of the input data for Mainframe-Attached platform as: • Variable-length text The SET RECORD command: • Can be specified only one time per Teradata FastLoad job script • When specified, must be appear before the DEFINE command Syntax Figure 1: For Mainframe-Attached Client Systems SET RECORD VARTEXT ‘|’ A ‘c’ DELIMITER A ; DISPLAY ERRORS ‘efilename’ NOSTOP TRIM NONE LEADING TRAILING BOTH QUOTE NO OPTIONAL YES 154 ‘p’ ‘ ” ’ ‘ q ’ ‘ r ’ 2411B001 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SET RECORD Figure 2: For Network-Attached Client Systems SET RECORD FORMATTED UNFORMATTED BINARY TEXT VARTEXT DELIMITER ; A 'I' 'c ' ; A DISPLAY_ERRORS 'efilename' NOSTOP TRIM QUOTE NONE LEADING TRAILING BOTH NO OPTIONAL YES 'p' '"' 'q' 'r' 2411A012 where Syntax Element Description FORMATTED Keyword specification that the input data source is in Teradata Database standard format. This is the default specification, if the SET RECORD command is not used in the Teradata FastLoad job script. UNFORMATTED Keyword specification that the input data source deviates from Teradata Database standard format. Unformatted records are any data file, such as a text file, that does not have various properties such as a consistent structure with regard to record length and order of data elements. BINARY Keyword specification that the input data source is in binary format. The format must be a 2-byte integer, n, followed by n bytes of data. TEXT Keyword specification that the input data source is in text format. The format must be an arbitrary number of bytes, followed by an end-of-record marker, which is a: • Line feed (x’0A) on UNIX platforms • Carriage-return/line feed pair (X’0D0A’) on Windows platforms VARTEXT Teradata FastLoad Reference Keyword specification that the input data source is in variable-length text record format, with each field separated by a delimiter character. 155 Chapter 3: Teradata FastLoad Commands SET RECORD Syntax Element Description ‘c’ Optional specification of the delimiter that separates fields in the variable-length text records of the input data source The delimiter can be a single or multi-character sequence (or string). If the delimiter is not specified, the default is the character sequence consists of a single pipe character (|). If the script character set is different from the client session character set, the delimiter is converted from the script character set to the client session character set before it is passed to Data Connector. Note: Any character sequence that appears in the data cannot be used as a delimiter. No control character other than a tab character can be used in a delimiter. DISPLAY_ERRORS Optional keyword specification that writes input data records that produce errors to the standard error file. 'efilename' Optional specification of a regular file name used to store erroneous variable-length text records. If it's specified, it must be specified after the DISPLAY_ERRORS keyword. If not specified, erroneous variable-length text records will be displayed on stderr. 156 NOSTOP Optional keyword specification that inhibits the Teradata FastLoad termination in response to an error condition associated with a variable-length text record. TRIM Optional keyword. It is used to specify whether field values in variable-length text record could be trimmed. It must be followed by one of the following keywords: NONE, LEADING, TRAILING or BOTH. NONE Can follow the keyword TRIM. It is used to specify that field values are not to be trimmed. TRIM NONE is the default behavior of the trim processing, which is the same as not specifying the TRIM at all. LEADING Can follow the keyword TRIM. It is used to specify the leading characters of field values must be trimmed. See 'p' below for trim character specification. TRAILING Can follow keyword TRIM. It is used to specify that the trailing characters of field values must be trimmed. See 'p' below for trim character specification. BOTH Can follow keyword TRIM. It is used to specify that the leading and trailing characters of field values must be trimmed. See 'p' below for trim character specification. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SET RECORD Syntax Element Description 'p' Optional specification of the trim character in field values of variable-length text records of the input data source. It is specified after the keyword LEADING, TRAILING or BOTH. Rules for a trim character are: • The trim character must be a single character, but may be either a single-byte or multi-byte character. It is expressed in the client session character set. • By default, if 'p' is not specified, the trim character is the blank (space) character. • Trimming can be performed on either unquoted or quoted field values. • If a field consists solely of one or more trim characters, it will be a zero-length VARCHAR after trimming. It can be set to NULL using NULLIF option on the DEFINE command. QUOTE Optional keyword. It is used to specify whether field values in variable-length text record will never be quoted (if it is followed by keyword NO), optionally be quoted (if it is followed by keyword OPTIONAL) or always be quoted (if it is followed by keyword YES). It must be followed by one of the following keywords: NO, OPTIONAL or YES. NO Can follow keyword QUOTE. It is used to specify that field values will never be quoted. It is the default behavior. OPTIONAL Can follow keyword QUOTE. It is used to specify that field values will optionally be quoted. YES Can follow keyword QUOTE. It is used to specify that field values will always be quoted. 'q' Optional specification of the opening quoted character in field values of variable-length text records of the input data source. See 'r' for more information. 'r' Optional specification of the closing quoted character in field values of variable-length text records of the input data source. Rules for opening and closing quoted characters are: • The quote character, either opening or closing quote, must be a single character, but may be either a single-byte or multi-byte character. It is expressed in the client session character set. • The opening and closing quote characters can be different. • If only 'q' is specified, it's used for both opening and closing quotes. • By default, if 'q' or 'r' are not specified, the opening quote or the closing quote is the '"' character. Usage Notes The following are the things to consider when using the SET RECORD command. • Teradata FastLoad Reference Data Formats 157 Chapter 3: Teradata FastLoad Commands SET RECORD The input source data can be either in text or binary format, where • • Text format is a data source containing characters for display on an ASCII terminal. • Binary format is numbers in hexadecimal Unformatted Records When UNFORMATTED is specified, Teradata FastLoad assumes nothing concerning the structure of the data, end-of-record delimiters, special characters and field length indicators. The input data can be either text or binary: • Use both an INSERT statement and a DEFINE command to define the fields • For binary data, manually insert the indicator bytes preceding each record Teradata FastLoad then uses the DEFINE clause as a guide to calculate the actual length of each record. Data that is extraneous and not intended for use can be defined as CHAR. Note: For ASCII data, line ending characters can differ from platform to platform. For example, some systems might only use a carriage return character, while others might use both a carriage return and a line feed character to end a line. Always consider the platform-dependent characteristics when reading ASCII data from a text file. • VARTEXT Records When VARTEXT is specified, Teradata FastLoad assumes that the input data is variable-length text fields separated by a field delimiter character. The utility parses each input data record on a field-by-field basis, and creates a VARCHAR field for each input text field. • Data Type Specifications When using the VARTEXT specification, VARCHAR and VARBYTE are the only valid data type specifications which can be used in the Teradata FastLoad DEFINE command. • Null Fields Two consecutive delimiter characters direct Teradata FastLoad to null the field corresponding to the one right after the first delimiter character. Also, if the last character in a record is a delimiter character, and yet there was at least one more field to be processed, Teradata FastLoad nulls the field corresponding to the next one to be processed, as defined in the DEFINE command. • Input Record Requirements The total number of fields in each input record must be equal to or greater than the number of fields described in the Teradata FastLoad DEFINE command. If the total number is less, Teradata FastLoad generates an error message. If the total number is more, the Teradata Database ignores the extra fields. • Error Record Handling When Teradata FastLoad encounters an error condition in an input record, it normally discards the record and terminates. When loading variable-length text records, either or both of these functions can be inhibited by specifying the error-handling options: • 158 DISPLAY_ERRORS Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SET RECORD • NOSTOP By specifying both options and redirecting STDERR to a file location instead of the terminal screen, the Teradata FastLoad job will run to completion and save all the error records. They can then be manually modified using another utility such as BTEQ or MultiLoad to load them into the table. • Variable-length Fields When using variable-length fields in either formatted or unformatted records, either: • Include a two-byte binary integer indicator immediately preceding each variable-length field. Teradata FastLoad uses this indicator to determine the exact length of the field. • Pad each variable-length field with blanks to produce fixed-length fields In either case, the maximum field length as shown in the table definition cannot be exceeded. • DEFINE and INSERT Specifications Use VARCHAR specifications in the DEFINE command and INSERT statements for variable-length data: User.Table Name Co1001 Co1002 Co1003 define Definition Type Size Integer 4 bytes Varchar(8) up to 8 bytes Date 4 bytes Co1001 (integer), Co1002 (Varchar(8)), Co1003 (date) file = file_path ; insert into User.Table values ( :Co1001, :Co1002, :Co1003 ) ; To pad a variable-length field to the maximum used in the table definition (in this case eight bytes) define column 2 as Char(8) with the table definition remaining Varchar(8). The following table (User.Table) contains three columns of fixed-length data types. Each record has four bytes as an integer, followed by eight bytes of characters, and then four bytes of a date in integer format: User.Table Definition Name Type Co1001 Integer Co1002 Char(8) Co1003 Date Size 4 bytes 8 bytes 4 bytes Assuming that the fields in the record correspond exactly to the table columns, the DEFINE command and INSERT statement specifications would be: define Co1001 (integer), Co1002 (char(8)), Co1003 (date) file = file_path ; insert into User.Table values ( :Co1001, :Co1002, Teradata FastLoad Reference 159 Chapter 3: Teradata FastLoad Commands SET RECORD :Co1003 ) ; Co1001 Co1002 Co1003 |00030506|4549474854202020|000CFD1F The DEFINE and INSERT specifications to define undesirable data (such as special control characters or carriage returns using HEX 0A as end-of-record delimiters) would be: defineDummy(char(8)), Co1001 (integer), Co1002 (char(8)), Co1003 (date), Newline(char(1)) file = file_path ; insert into User.Table values ( :Co1001, :Co1002, :Co1003 ) ; Control Char Co1001 Co1002 Co1003 0FCA037CB86BFF8A|00030506|4549474854202020|000CFD1F|0A New line Note: Some systems might require a Newline(char(2)) specification instead of Newline(char(1)). Example The following command example sets records to unformatted mode: set record unformatted ; Completion Message The Teradata FastLoad completion message is: Now set to read “UNFORMATTED” records. 160 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SET SESSION CHARSET SET SESSION CHARSET Purpose The SET SESSION CHARSET command specifies which character set is in effect during a specific Teradata FastLoad invocation. Note: The SET SESSION CHARSET command applies only to network-attached systems. For mainframe-attached systems, see “Invoking Teradata FastLoad” on page 33 for a description of the CHARSET parameter. Syntax SET SESSION CHARSET " ASCII " ; KANJIEUC _ 0 U KANJISJIS _ 0 S 2411B029 where Syntax Element Description ASCII ASCII character set. KANJIEUC_0U Kanji Extended UNIX Code character set. KANJISJIS_0S Combined JIS-x0208 and JIS-x0201 character set developed by Microsoft. Usage Notes Table 50 describes the things to consider when using the SET SESSION CHARSET command. Table 50: Usage Notes for SET SESSION CHARSET Topic Usage Notes Command Placement The SET SESSION CHARSET command must appear before the LOGON command in the Teradata FastLoad job script. Double Quote Characters in Character Set Name Specifications The names of the character sets must be enclosed in double quote characters. Teradata FastLoad Reference 161 Chapter 3: Teradata FastLoad Commands SET SESSION CHARSET Table 50: Usage Notes for SET SESSION CHARSET (continued) Topic Usage Notes Priority of Character Set Specifications The order in which the character set is determined for a Teradata FastLoad job is: 1 User specified by a: • Runtime parameter (This specification has the highest priority on all supported platforms.) • SET SESSION CHARSET command on network-attached systems • CHARSET parameter in the Teradata FastLoad configuration file 2 System Parameter Block (SPB) specified by the: • HSHSPB on mainframe-attached systems • clispb.dat file on network-attached systems. 3 Teradata Database default, determined by a query from Teradata FastLoad. 162 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SHOW SHOW Purpose The SHOW command displays active definitions for the input data source or INMOD routine and the field names that were established by one or more DEFINE commands. Syntax SHOW ; 2411A030 Usage Notes The SHOW command displays: • Field names • Data type (integer or character) of each field • Internal length of each field • Offset of each field within the input record • Input data source or INMOD routine names If a DEFINE command has not been entered, the result is blank and Teradata FastLoad responds with this message: TOTAL RECORD LENGTH = 0 Note: The SHOW command is processed by Teradata FastLoad. It is not transmitted to the Teradata Database. Example When the following DEFINE command is active: DEFINE EmpNo (smallint), Name (varchar(12)), DeptNo (decimal (3,0)), JobTitle (varchar(12)), Salary (decimal(8,2)), YrsExp (byteint), DOB (date), Sex (char(1)), Race (char(1)), MStat (char(1)), EdLev (byteint), HCap (byteint) FILE=EmpData ; The Teradata FastLoad response to the SHOW command is: Teradata FastLoad Reference 163 Chapter 3: Teradata FastLoad Commands SHOW FILE = EmpData EMPNO OFFSET= NAME OFFSET = DEPTNO OFFSET= JOBTITLE OFFSET= SALARY OFFSET= YRSEXP OFFSET= DOB OFFSET= SEX OFFSET= RACE OFFSET= MSTAT OFFSET= EDLEV OFFSET= HCAP OFFSET= TOTAL RECORD LENGTH= 164 0 2 16 18 32 36 37 41 42 43 44 45 46 LEN LEN LEN LEN LEN LEN LEN LEN LEN LEN LEN LEN = = = = = = = = = = = = 2 12 2 12 4 1 4 1 1 1 1 1 SMALLINT VARCHAR DECIMAL VARCHAR DECIMAL BYTEINT DATE CHAR CHAR CHAR BYTEINT BYTEINT Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SHOW VERSIONS SHOW VERSIONS Purpose The SHOW VERSIONS command displays the current level of all Teradata FastLoad utility software modules. Syntax SHOW VERSIONS ; VERSION 2411A031 Usage Notes Use the SHOW VERSIONS command to retrieve the version information when reporting software problems. Example The following command example displays the current versions of software modules in use: .show version; 0001 .show version; FastLoad Version 14.10.00.00 for Win 32 running Windows Sockets FastLoad : 14.10.00.05 FastCmds : 14.10.00.08 FastIO : 14.10.00.00 FastMBCS : 14.10.00.00 FastNtfy : 14.00.00.03 FastPars : 14.10.00.01 FastSQL : 14.10.00.12 FastUtil : 14.10.00.01 Fdlosdep : 13.01.00.00 Teradata Data Connector : 14.10.00.00 PMPROCS : 14.10.00.05 PMRWFMT : 14.10.00.01 PMTRCE : 13.10.00.02 PMMM : 13.00.00.01 PMUDDI : 14.10.00.03 DCUDDI : 14.10.00.09 PMHEXDMP : 13.10.00.01 PMUNXDSK : 14.10.00.05 ICUVER : TDICU, 14.10.00.00 CLIV2 : 14.10.00.15 MTDP : 14.10.00.12 MOSIos : 14.10.00.01 MOSIDEP : 14.00.00.04 OSENCRYPT : N/A OSERR : 14.00.00.00 Teradata FastLoad Reference 165 Chapter 3: Teradata FastLoad Commands SHOW VERSIONS FastLoad linking date: Aug 16 2012 166 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SLEEP SLEEP Purpose The SLEEP command specifies the number of minutes that Teradata FastLoad pauses before retrying a logon operation when the maximum number of load operations is already running on the Teradata Database. Syntax ; SLEEP minutes 2411A032 where Syntax Element Description minutes Number of minutes that Teradata FastLoad pauses before retrying the logon operation. The minutes specification must be greater than zero. If zero is entered, Teradata FastLoad responds with an error message, and terminates. The Teradata FastLoad default, if the SLEEP command is not used, the number of minutes is 6. Usage Notes Table 51 describes the things to consider when using the SLEEP command. Teradata FastLoad Reference 167 Chapter 3: Teradata FastLoad Commands SLEEP Table 51: Usage Notes for SLEEP Topic Usage Notes Function The SLEEP specification works with the TENACITY specification to control Teradata FastLoad logon attempts. When Teradata FastLoad tries to log on for a new session, and the Teradata Database indicates that the maximum number of load sessions is already running, the Teradata FastLoad utility: 1 Logs off any new sessions that were logged on 2 Waits for 6 minutes, by default, or for the amount of time specified by the SLEEP command However, if the amount of time specified by the SLEEP command exceeds that of the TENACITY command, then the sleep interval is reset and equated to the amount of time specified by the TENACITY command. For example, if the time specified with the SLEEP command is 65 minutes and the time specified with TENACITY command is 1 hour, then the SLEEP time is reset to 60 minutes so that the SLEEP time does not exceed the TENACITY time. 3 Tries again to log on to the Teradata Database Teradata FastLoad repeats this process until it has either logged on for the required number of sessions or equates the amount of time specified by the TENACITY command. Note: The sleep interval specified in the SLEEP command is dynamically adjusted so that the total sleep times does not exceed the amount of time specified by the TENACITY command. For example, if the time specified with the SLEEP command is 35 minutes and the time specified with the TENACTY command is 1 hour, then: • FastLoad sleeps for 35 minutes and then attempts to log on to the Teradata Database. • If the first logon attempt fails, then the SLEEP time is adjusted to 25 minutes so that the total SLEEP time does not exceed the TENACITY time. Command Placement 168 Command placement affects the logon operation. State the SLEEP and TENACITY commands before the LOGON command in the Teradata FastLoad job script. Teradata FastLoad terminates with an error message if these commands are stated after the LOGON command. Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands SLEEP Table 51: Usage Notes for SLEEP (continued) Topic Usage Notes Command Overrides The SLEEP and TENACITY command specifications override the corresponding SLEEP and TENACITY specifications that may be made in the Teradata FastLoad configuration file. Similarly, the SLEEP and TENACITY command specifications themselves are overridden by the corresponding specifications made as runtime parameters when invoking Teradata FastLoad. The order of preferences for the SLEEP and TENACITY specifications, from highest to lowest, is: 1 Runtime parameters 2 Teradata FastLoad script commands 3 Teradata FastLoad configuration file specifications 4 Teradata FastLoad default values Teradata FastLoad Reference 169 Chapter 3: Teradata FastLoad Commands TENACITY TENACITY Purpose The TENACITY command specifies the number of hours that Teradata FastLoad continues trying to log on when the maximum number of load operations is already running on the Teradata Database. Syntax ; TENACITY hours 2411A033 where Syntax Element Description hours Number of hours that Teradata FastLoad continues trying to log on. The hours specification must be greater than zero. If zero is entered, Teradata FastLoad responds with an error message and terminates. Usage Notes Table 52 describes the things to consider when using the TENACITY command. 170 Teradata FastLoad Reference Chapter 3: Teradata FastLoad Commands TENACITY Table 52: Usage Notes for TENACITY Topic Usage Notes Function The TENACITY specification works with the SLEEP specification to control Teradata FastLoad logon attempts. When Teradata FastLoad tries to log on for a new session, and the Teradata Database indicates that the maximum number of load sessions is already running, Teradata FastLoad: 1 Logs off any new sessions that were logged on 2 Waits for 6 minutes, by default, or for the amount of time specified by the SLEEP command 3 Tries again to log on to the Teradata Database Teradata FastLoad repeats this process until it has either logged on for the required number of sessions or exceeded the amount of time specified by the TENACITY command. For more information on how the TENACITY command interacts with the SLEEP command, please see Table 51 on page 168. Note: The utility default for TENACITY is no tenacity. A Teradata FastLoad configuration file entry, the runtime parameter, or a TENACITY command must be used in the Teradata FastLoad job script to enable the tenacity feature for the Teradata FastLoad logon operation. Command Placement Command placement affects the logon operation. State the TENACITY and SLEEP commands before the LOGON command in the Teradata FastLoad job script. Teradata FastLoad terminates with an error message if the Teradata FastLoad job script states the TENACITY or SLEEP command after the LOGON command. Command Overrides The TENACITY and SLEEP command specifications override the corresponding TENACITY and SLEEP specifications that may be made in the Teradata FastLoad configuration file. Similarly, the TENACITY and SLEEP command specifications themselves are overridden by the corresponding specifications made as runtime parameters when invoking Teradata FastLoad. The order of preferences for the TENACITY and SLEEP specifications, from highest to lowest, is: 1 Runtime parameters 2 Teradata FastLoad script commands 3 Teradata FastLoad configuration file specifications 4 Teradata FastLoad default values Teradata FastLoad Reference 171 Chapter 3: Teradata FastLoad Commands TENACITY 172 Teradata FastLoad Reference CHAPTER 4 Troubleshooting This chapter provides a description of user aids for identifying and correcting errors that might occur during a Teradata FastLoad task. Foremost among these tools are a large number of error messages. For more information on error messages, see Messages (B035-1096). Troubleshooting information in this chapter includes: • Compiler Versions Compiler Versions Table 53 lists compilers and the version used while building the utility. The C runtime libraries that support these compiler versions must be installed on the client system. Table 53: Compiler Versions Platform Compiler Version AIX IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) Version: 12.01.0000.0000 HPUX-IA64 HP C/aC++ B3910B A.06.20 [May 13 2008] HPUX-PARISC HP92453-01 B.11.11.16 HP C Compiler SOLARIS SPARC Sun C 5.8 2005/10/13 SOLARIS OPTERON Sun C 5.8 Patch 121016-08 2009/04/20 LINUX gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux) z/Linux gcc version 4.1.0 (SUSE Linux) WINDOWS Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 Mainframe z/OS V1.11 XL C/C++ Teradata FastLoad Reference 173 Chapter 4: Troubleshooting Compiler Versions 174 Teradata FastLoad Reference APPENDIX A How to Read Syntax Diagrams This appendix describes the conventions that apply to reading the syntax diagrams used in this book. Syntax Diagram Conventions Notation Conventions Item Definition / Comments Letter An uppercase or lowercase alphabetic character ranging from A through Z. Number A digit ranging from 0 through 9. Do not use commas when typing a number with more than 3 digits. Word Keywords and variables. • UPPERCASE LETTERS represent a keyword. Syntax diagrams show all keywords in uppercase, unless operating system restrictions require them to be in lowercase. • lowercase letters represent a keyword that you must type in lowercase, such as a Linux command. • lowercase italic letters represent a variable such as a column or table name. Substitute the variable with a proper value. • lowercase bold letters represent an excerpt from the diagram. The excerpt is defined immediately following the diagram that contains it. • UNDERLINED LETTERS represent the default value. This applies to both uppercase and lowercase words. Spaces Use one space between items such as keywords or variables. Punctuation Type all punctuation exactly as it appears in the diagram. Paths The main path along the syntax diagram begins at the left with a keyword, and proceeds, left to right, to the vertical bar, which marks the end of the diagram. Paths that do not have an arrow or a vertical bar only show portions of the syntax. The only part of a path that reads from right to left is a loop. Teradata FastLoad Reference 175 Appendix A: How to Read Syntax Diagrams Syntax Diagram Conventions Continuation Links Paths that are too long for one line use continuation links. Continuation links are circled letters indicating the beginning and end of a link: A A FE0CA002 When you see a circled letter in a syntax diagram, go to the corresponding circled letter and continue reading. Required Entries Required entries appear on the main path: SHOW FE0CA003 If you can choose from more than one entry, the choices appear vertically, in a stack. The first entry appears on the main path: SHOW CONTROLS VERSIONS FE0CA005 Optional Entries You may choose to include or disregard optional entries. Optional entries appear below the main path: SHOW CONTROLS 176 FE0CA004 Teradata FastLoad Reference Appendix A: How to Read Syntax Diagrams Syntax Diagram Conventions If you can optionally choose from more than one entry, all the choices appear below the main path: READ SHARE JC01A010 ACCESS Some commands and statements treat one of the optional choices as a default value. This value is UNDERLINED. It is presumed to be selected if you type the command or statement without specifying one of the options. Strings String literals appear in apostrophes: 'msgtext ' JC01A004 Abbreviations If a keyword or a reserved word has a valid abbreviation, the unabbreviated form always appears on the main path. The shortest valid abbreviation appears beneath. SHOW CONTROLS CONTROL FE0CA042 In the above syntax, the following formats are valid: • SHOW CONTROLS • SHOW CONTROL Loops A loop is an entry or a group of entries that you can repeat one or more times. Syntax diagrams show loops as a return path above the main path, over the item or items that you can repeat: , , ( cname 3 4 ) JC01B012 Teradata FastLoad Reference 177 Appendix A: How to Read Syntax Diagrams Syntax Diagram Conventions Read loops from right to left. The following conventions apply to loops: IF... THEN... there is a maximum number of entries allowed the number appears in a circle on the return path. there is a minimum number of entries required the number appears in a square on the return path. a separator character is required between entries the character appears on the return path. In the example, you may type cname a maximum of 4 times. In the example, you must type at least three groups of column names. If the diagram does not show a separator character, use one blank space. In the example, the separator character is a comma. a delimiter character is required around entries the beginning and end characters appear outside the return path. Generally, a space is not needed between delimiter characters and entries. In the example, the delimiter characters are the left and right parentheses. Excerpts Sometimes a piece of a syntax phrase is too large to fit into the diagram. Such a phrase is indicated by a break in the path, marked by (|) terminators on each side of the break. The name for the excerpted piece appears between the terminators in boldface type. The boldface excerpt name and the excerpted phrase appears immediately after the main diagram. The excerpted phrase starts and ends with a plain horizontal line: LOCKING excerpt A A HAVING con excerpt where_cond , cname , col_pos JC01A014 178 Teradata FastLoad Reference Appendix A: How to Read Syntax Diagrams Syntax Diagram Conventions Multiple Legitimate Phrases In a syntax diagram, it is possible for any number of phrases to be legitimate: dbname DATABASE tname TABLE vname VIEW JC01A016 In this example, any of the following phrases are legitimate: • dbname • DATABASE dbname • tname • TABLE tname • vname • VIEW vname Sample Syntax Diagram , CREATE VIEW viewname AS A LOCKING cname CV LOCK ACCESS dbname A DATABASE tname FOR SHARE IN READ TABLE WRITE EXCLUSIVE vname VIEW EXCL , B SEL B MODE expr , FROM qual_cond tname C .aname C HAVING cond ; qual_cond , WHERE cond GROUP BY cname , col_pos JC01A018 Teradata FastLoad Reference 179 Appendix A: How to Read Syntax Diagrams Syntax Diagram Conventions Diagram Identifier The alphanumeric string that appears in the lower right corner of every diagram is an internal identifier used to catalog the diagram. The text never refers to this string. 180 Teradata FastLoad Reference APPENDIX B Multifile Teradata FastLoad Job Script Examples This appendix provides three example multifile Teradata FastLoad job scripts. The third script includes the END LOADING command. The following subsections show the output from the three Teradata FastLoad job scripts. Note: The third output file contains the final statistics. First Output File =================================================================== = = = FASTLOAD UTILITY VERSION 14.10.00.00 = = PLATFORM WIN32 = = = =================================================================== =================================================================== = = = Copyright 1984-2012, Teradata Corporation. = = ALL RIGHTS RESERVED. = = = =================================================================== **** 11:20:51 Processing starting at: Fri Sep 21 11:20:51 2012 0001 .SESSIONS 2; **** 11:20:51 FDL4866 SESSIONS command accepted =================================================================== = = = Logon/Connection = = = =================================================================== 0002 .logon 153.65.168.87/weekly, **** **** **** **** **** 11:20:51 11:20:51 11:20:51 11:20:51 11:20:51 Teradata Database Release: 14.10.00.99 Teradata Database Version: 14.10.00.99 Number of AMPs available: 4 Current CLI or RDBMS allows maximum row size: 64K Character set for this job: ASCII Teradata FastLoad Reference 181 Appendix B: Multifile Teradata FastLoad Job Script Examples First Output File 0003 .SHOW VERSION; FastLoad Version 14.10h.00.00 for Win 32 running Windows Sockets FastLoad : 14.10.00.05 FastCmds : 14.10.00.08 FastIO : 14.10.00.00 FastMBCS : 14.10.00.00 FastNtfy : 14.00.00.03 FastPars : 14.10.00.01 FastSQL : 14.10.00.12 FastUtil : 14.10.00.01 Fdlosdep : 13.01.00.00 Teradata Data Connector : 14.10.00.00 PMPROCS : 14.10.00.05 PMRWFMT : 14.10.00.01 PMTRCE : 13.10.00.02 PMMM : 13.00.00.01 PMUDDI : 14.10.00.03 DCUDDI : 14.10.00.09 PMHEXDMP : 13.10.00.01 PMUNXDSK : 14.10.00.05 ICUVER : TDICU, 14.10.00.00 CLIV2 : 14.10.00.15 MTDP : 14.10.00.12 MOSIos : 14.10.00.01 MOSIDEP : 14.00.00.04 OSENCRYPT : N/A OSERR : 14.00.00.00 FastLoad linking date: Aug 16 2012 0004 DROP TABLE FL0020e1; **** 11:20:52 RDBMS error 3807: Object 'FL0020e1' does not exist. 0005 DROP TABLE FL0020e2; **** 11:20:52 RDBMS error 3807: Object 'FL0020e2' does not exist. 0006 DROP TABLE FL0020; **** 11:20:52 Command completed successfully 0007 RECORD 1 THRU 10; **** 11:20:52 Starting record number set to **** 11:20:52 Ending record number set to : 1 : 10 0008 create table FL0020 (a int, b char(10)); **** 11:20:52 Command completed successfully 0009 DEFINE FIELD1 (integer), FIELD2 (char(10)), FILE=FDAT02; 182 Teradata FastLoad Reference Appendix B: Multifile Teradata FastLoad Job Script Examples First Output File **** 11:20:53 FDL4803 DEFINE statement processed 0010 BEGIN LOADING FL0020 ERRORFILES FL0020e1,FL0020e2 checkpoint 100; **** **** **** **** **** 11:20:53 11:20:53 11:20:53 11:20:53 11:20:53 Number of FastLoad sessions requested = 2 Number of FastLoad sessions connected = 2 FDL4808 LOGON successful Number of AMPs available: 4 BEGIN LOADING COMPLETE 0011 SHOW; FILE = FDAT02 FIELD1 FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = OFFSET = 0 LEN = 4 LEN = 4 INTEGER 10 CHAR =================================================================== = = = Insert Phase = = = =================================================================== 0012 INSERT INTO FL0020(A,B) VALUES (:field1,:FIELD2); **** **** **** **** 11:20:53 11:20:53 11:20:53 11:20:53 Number of recs/msg: 3382 Starting to send to RDBMS with record 1 Sending row 10 Finished sending rows to the RDBMS **** 11:20:53 Acquisition Phase statistics: Elapsed time: 00:00:00 (in hh:mm:ss) CPU time: 0.0156001 Seconds MB/sec: N/A MB/cpusec: 0.01 0013 SHOW; FILE = FDAT02 FIELD1 FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = OFFSET = 0 LEN = 4 LEN = 4 INTEGER 10 CHAR 0014 LOGOFF; =================================================================== = = = Logoff/Disconnect = = = =================================================================== **** 11:20:54 Logging off all **** 11:20:55 Total processor . Start : Fri Sep . End : Fri Sep Teradata FastLoad Reference sessions time used = '0.171601 Seconds' 21 11:20:51 2012 21 11:20:55 2012 183 Appendix B: Multifile Teradata FastLoad Job Script Examples Second Output File . Highest return code encountered = '4'. **** 11:20:55 FastLoad Paused Second Output File =================================================================== = = = FASTLOAD UTILITY VERSION 14.10.00.00 = = PLATFORM WIN32 = = = =================================================================== =================================================================== = = = Copyright 1984-2012, Teradata Corporation. = = ALL RIGHTS RESERVED. = = = =================================================================== **** 11:23:09 Processing starting at: Fri Sep 21 11:23:09 2012 0001 .SESSIONS 10; **** 11:23:09 FDL4866 SESSIONS command accepted =================================================================== = = = Logon/Connection = = = =================================================================== 0002 .logon 153.65.168.87/weekly, **** **** **** **** **** 11:23:09 11:23:09 11:23:09 11:23:09 11:23:09 Teradata Database Release: 14.10.00.99 Teradata Database Version: 14.10.00.99 Number of AMPs available: 4 Current CLI or RDBMS allows maximum row size: 64K Character set for this job: ASCII 0003 .SHOW VERSION; FastLoad Version 14.10h.00.00 for Win 32 running Windows Sockets FastLoad : 14.10.00.05 FastCmds : 14.10.00.08 FastIO : 14.10.00.00 FastMBCS : 14.10.00.00 FastNtfy : 14.00.00.03 FastPars : 14.10.00.01 FastSQL : 14.10.00.12 FastUtil : 14.10.00.01 Fdlosdep : 13.01.00.00 Teradata Data Connector : 14.10.00.00 PMPROCS : 14.10.00.05 PMRWFMT : 14.10.00.01 PMTRCE : 13.10.00.02 184 Teradata FastLoad Reference Appendix B: Multifile Teradata FastLoad Job Script Examples Second Output File PMMM PMUDDI DCUDDI PMHEXDMP PMUNXDSK ICUVER CLIV2 MTDP MOSIos MOSIDEP OSENCRYPT OSERR : : : : : : : : : : : : 13.00.00.01 14.10.00.03 14.10.00.09 13.10.00.01 14.10.00.05 TDICU, 14.10.00.00 14.10.00.15 14.10.00.12 14.10.00.01 14.00.00.04 N/A 14.00.00.00 FastLoad linking date: Aug 16 2012 0004 DROP TABLE FL0020e1; **** 11:23:09 Command completed successfully 0005 DROP TABLE FL0020e2; **** 11:23:09 Command completed successfully 0006 DROP TABLE FL0020; **** 11:23:09 Command completed successfully 0007 RECORD 201 THRU 210; **** 11:23:09 Starting record number set to **** 11:23:09 Ending record number set to : 201 : 210 0008 create table FL0020 (a int, b char(10)); **** 11:23:09 Command completed successfully 0009 DEFINE FIELD1 (integer), FIELD2 (char(10)), FILE=FDAT02; **** 11:23:09 FDL4803 DEFINE statement processed 0010 BEGIN LOADING FL0020 ERRORFILES FL0020e1,FL0020e2 checkpoint 100; **** **** **** **** **** 11:23:10 11:23:10 11:23:10 11:23:10 11:23:10 Number of FastLoad sessions requested = 10 Number of FastLoad sessions connected = 4 FDL4808 LOGON successful Number of AMPs available: 4 BEGIN LOADING COMPLETE 0011 SHOW; FILE = FDAT02 FIELD1 Teradata FastLoad Reference OFFSET = 0 LEN = 4 INTEGER 185 Appendix B: Multifile Teradata FastLoad Job Script Examples Third Output File FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = 4 LEN = 10 CHAR =================================================================== = = = Insert Phase = = = =================================================================== 0012 INSERT INTO FL0020(A,B) VALUES (:field1,:FIELD2); **** **** **** **** 11:23:10 11:23:10 11:23:10 11:23:10 Number of recs/msg: 3382 Starting to send to RDBMS with record 201 Sending row 210 Finished sending rows to the RDBMS **** 11:23:10 Acquisition Phase statistics: Elapsed time: 00:00:00 (in hh:mm:ss) CPU time: 0.0156001 Seconds MB/sec: N/A MB/cpusec: 0.01 0013 SHOW; FILE = FDAT02 FIELD1 FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = OFFSET = 0 LEN = 4 LEN = 4 INTEGER 10 CHAR 0014 LOGOFF; =================================================================== = = = Logoff/Disconnect = = = =================================================================== **** 11:23:10 Logging off all sessions **** 11:23:12 Total processor time used = '0.343202 Seconds' . Start : Fri Sep 21 11:23:09 2012 . End : Fri Sep 21 11:23:12 2012 . Highest return code encountered = '4'. **** 11:23:12 FastLoad Paused Third Output File =================================================================== = = = FASTLOAD UTILITY VERSION 14.10.00.00 = = PLATFORM WIN32 = = = =================================================================== =================================================================== 186 Teradata FastLoad Reference Appendix B: Multifile Teradata FastLoad Job Script Examples Third Output File = = = Copyright 1984-2012, Teradata Corporation. = = ALL RIGHTS RESERVED. = = = =================================================================== **** 12:12:03 Processing starting at: Fri Sep 21 12:12:03 2012 0001 .SESSIONS 10; **** 12:12:03 FDL4866 SESSIONS command accepted =================================================================== = = = Logon/Connection = = = =================================================================== 0002 .logon 153.65.168.87/weekly, **** **** **** **** **** 12:12:03 12:12:03 12:12:03 12:12:03 12:12:03 Teradata Database Release: 14.10.00.99 Teradata Database Version: 14.10.00.99 Number of AMPs available: 4 Current CLI or RDBMS allows maximum row size: 64K Character set for this job: ASCII 0003 .SHOW VERSION; FastLoad Version 14.10h.00.00 for Win 32 running Windows Sockets FastLoad : 14.10.00.05 FastCmds : 14.10.00.08 FastIO : 14.10.00.00 FastMBCS : 14.10.00.00 FastNtfy : 14.00.00.03 FastPars : 14.10.00.01 FastSQL : 14.10.00.12 FastUtil : 14.10.00.01 Fdlosdep : 13.01.00.00 Teradata Data Connector : 14.10.00.00 PMPROCS : 14.10.00.05 PMRWFMT : 14.10.00.01 PMTRCE : 13.10.00.02 PMMM : 13.00.00.01 PMUDDI : 14.10.00.03 DCUDDI : 14.10.00.09 PMHEXDMP : 13.10.00.01 PMUNXDSK : 14.10.00.05 ICUVER : TDICU, 14.10.00.00 CLIV2 : 14.10.00.15 MTDP : 14.10.00.12 MOSIos : 14.10.00.01 MOSIDEP : 14.00.00.04 OSENCRYPT : N/A OSERR : 14.00.00.00 FastLoad linking date: Aug 16 2012 0004 DROP TABLE FL0020e1; Teradata FastLoad Reference 187 Appendix B: Multifile Teradata FastLoad Job Script Examples Third Output File **** 12:12:04 Command completed successfully 0005 DROP TABLE FL0020e2; **** 12:12:04 Command completed successfully 0006 DROP TABLE FL0020; **** 12:12:04 Command completed successfully 0007 RECORD 201 THRU 210; **** 12:12:04 Starting record number set to **** 12:12:04 Ending record number set to : 201 : 210 0008 create table FL0020 (a int, b char(10)); **** 12:12:04 Command completed successfully 0009 DEFINE FIELD1 (integer), FIELD2 (char(10)), FILE=FDAT02; **** 12:12:04 FDL4803 DEFINE statement processed 0010 BEGIN LOADING FL0020 ERRORFILES FL0020e1,FL0020e2 checkpoint 100; **** **** **** **** **** 12:12:04 12:12:04 12:12:04 12:12:05 12:12:05 Number of FastLoad sessions requested = 10 Number of FastLoad sessions connected = 4 FDL4808 LOGON successful Number of AMPs available: 4 BEGIN LOADING COMPLETE 0011 SHOW; FILE = FDAT02 FIELD1 FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = OFFSET = 0 LEN = 4 LEN = 4 INTEGER 10 CHAR =================================================================== = = = Insert Phase = = = =================================================================== 0012 INSERT INTO FL0020(A,B) VALUES (:field1,:FIELD2); **** **** **** **** 188 12:12:05 12:12:05 12:12:05 12:12:05 Number of recs/msg: 3382 Starting to send to RDBMS with record 201 Sending row 210 Finished sending rows to the RDBMS Teradata FastLoad Reference Appendix B: Multifile Teradata FastLoad Job Script Examples Third Output File **** 12:12:05 Acquisition Phase statistics: Elapsed time: 00:00:00 (in hh:mm:ss) CPU time: 0 Seconds MB/sec: N/A MB/cpusec: N/A 0013 SHOW; FILE = FDAT02 FIELD1 FIELD2 TOTAL RECORD LENGTH = 14 OFFSET = OFFSET = 0 LEN = 4 LEN = 4 INTEGER 10 CHAR =================================================================== = = = End Loading Phase = = = =================================================================== 0014 END LOADING; **** 12:12:05 END LOADING COMPLETE Total Records Read - skipped by RECORD command - sent to the RDBMS Total Error Table 1 Total Error Table 2 Total Inserts Applied Total Duplicate Rows Start: End : = = = = = = = 210 200 10 0 ---- Table has been dropped 0 ---- Table has been dropped 10 0 Fri Sep 21 12:12:05 2012 Fri Sep 21 12:12:05 2012 **** 12:12:05 Application Phase statistics: Elapsed time: 00:00:00 (in hh:mm:ss) 0015 LOGOFF; =================================================================== = = = Logoff/Disconnect = = = =================================================================== **** 12:12:05 Logging off all sessions **** 12:12:06 Total processor time used = '0.249602 Seconds' . Start : Fri Sep 21 12:12:03 2012 . End : Fri Sep 21 12:12:06 2012 . Highest return code encountered = '0'. **** 12:12:06 FDL4818 FastLoad Terminated Teradata FastLoad Reference 189 Appendix B: Multifile Teradata FastLoad Job Script Examples Third Output File 190 Teradata FastLoad Reference APPENDIX C INMOD and Notify Exit Routine Examples This appendix provides program listings of sample INMOD and notify exit routines for the following Teradata client platforms: • • • z/OS – example INMOD routines written in: • Assembler • COBOL • IBM C • PL/I UNIX OS – the sample INMOD routines that are provided with Teradata FastLoad software: • BLKEXIT.C • BLKEXITR.C All Platforms – the notify exit routine: flnfyext.c z/OS Assembler INMOD Example The INMOD in the following example reads a record from the input data file and adds a four-byte integer field to the front of the record. The new field contains a sequence record that ranges from one to the total number of input records. BULKCON TITLE’-- CONCATENATE INPUT RECORDS FOR INPUT TO FASTLOAD ’ BULKCON CSECT USING BULKCON,15 ******************************************************************** * THIS PROGRAM IS CALLED BY THE TERADATA FASTLOAD PROGRAMS * * TO OBTAIN A RECORD TO BE USED TO INSERT, UPDATE, DELETE, * * OR SELECT ROWS OF A DBC TABLE. * * * * THIS PROGRAM IS NOT REENTRANT. * * FUNCTION: * * READ AN INPUT RECORD AND ADD A FOUR-BYTE INTEGER FIELD * * THE FRONT OF THE RECORD. THE NEW FIELD WILL CONTAIN * * A SEQUENCE NUMBER WHICH RANGES FROM 1 TO ... * * NUMBER-OF-INPUT-RECORDS. * Teradata FastLoad Reference 191 Appendix C: INMOD and Notify Exit Routine Examples z/OS * * * RETURN TO THE CALLER, FASTLOAD, INDICATING * * EITHER MORE RECORDS ARE AVAILABLE OR NO MORE RECORDS * * ARE TO BE PROCESSED. * * * * THIS INMOD PROGRAM CAN BE USED TO ENSURE UNIQUE RECORDS * * IN CERTAIN APPLICATIONS, THE SEQUENCE FIELD * * CAN BE USED FOR “DATA SAMPLING”. * * * * DDNAME OF THE INPUT DATA SET: “INDATA” * * * ******************************************************************** B STOREGS BRANCH AROUND EP DC AL1(31) DEFINE EP LENGTH DC CL9’BULKIN ’ DEFINE DC CL9’&SYSDATE’ ENTRY DC CL8’ MVS ’ POINT DC CL5’&SYSTIME’ IDENTIFIER ******************************************************************** * SAVE REGISTERS * ******************************************************************** STOREGS DS 0H DEFINE AND ALIGN SYMBOL STM R14,R12,12(R13) STORE OFF CALLER’S REGISTERS LR R12,R15 COPY BASE ADDRESS DROP R15 DROP VOLATILE BASE REGISTER USING BULKCON,R12 ESTAB PERM CSECT ADDRBLTY LA R14,SAVEAREA POINT AT LOCAL SAVE WORK ST R14,8(,R13) STORE FWD LINK IN SA CHAIN ST R13,4(,R14) STORE BWD LINK IN SA CHAIN LR R13,R14 COPY LOCAL SAVE/WORK AREA ADDR L R11,0(,R1) POINT TO PARM SPACE 1 ******************************************************************** * VOPEN “DATA” DATA SET * * (ONLY THE FIRST TIME) * ******************************************************************** USING PREBUF,R11 COVER PRE-PROC AREA LA R9,PREREC POINT TO START OF PREPROC. DATA L R15,PRECODE CHECK ON CODE FROM FASTLOAD OC PRECODE,PRECODE FIRST ENTRY ? (0=FIRST ENTRY) BNZ NOOPEN NO, SKIP OPEN USING IHADCB,R10 YES,COVER DCB FOR OPEN LA R10,INDATA POINT TO DATA DCB OPEN INDATA OPEN INPUT DATA SET TM DCBOFLGS,X’10’ DID IT OPEN ? BO OPENOK YES, WTO ’UNABLE TO OPEN INDATADATA SET’,ROUTCDE=11 B BADRET RETURN WITH ERROR CODE ******************************************************************* * CHECK FASTLOAD/BULKLOAD STATUS CODES * * 0 = FIRST ENTRY (FASTLOAD/BULKLOAD EXPECTS TO RECEIVE A * * RECORD) * * 1 = GET NEXT RECORD (FASTLOAD/BULKLOAD EXPECTS TO RECEIVE A * * RECORD) * * 2 = Client RESTART CALL (FASTLOAD DOES NOT EXPECT A RECORD) * * 3 = CHECKPOINT CALL (FASTLOAD DOES NOT EXPECT A RECORD) * * 4 = DBC RESTART CALL (FASTLOAD DOES NOT EXPECT A RECORD) * * * * * 192 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples z/OS * NOTE: THIS INMOD WAS WRITTEN TO BE COMPATIBLE WITH FASTLOAD * * AND BULKLOAD AND THEREFORE CONTAINS INFORMATION ON * * STATUS CODES 2,3, AND 4 WHICH PERTAIN ONLY TO * * FASTLOAD. CODES 2,3, AND 4 ARE NOT HANDLED BY * * THIS PROGRAM. * ******************************************************************* OPENOK DS 0H MVI ISPOPEN,1 INDICATE INPUT FILE HAS BEEN OPEN NOOPEN L R15,PRECODE CHECK ON CODE FROM FASTLOAD C R15,=F’1’ NEED RECORD ? BH NOREC NO , DO NOT “GET” A RECORD L R15,SAMPNUM GET CURRENT SAMPLE NUM. LA R15,1(R15) INCR BY 1 ST R15,0(R9) STORE AT FRONT OF RECORD ST R15,SAMPNUM RESET COUNTER LA R9,4(R9) ADVANCE FOR READ ADDR. LA R10,INDATA COVER INDATA DCB GETNEXT GET INDATA,(R9) READ A RECORD INCREC LH R9,DCBLRECL GET RECORD LENGTH AH R9,=H’4’ ADD 4 FOR NEW FIELD SR R15,R15 SET RETURN CODE VALUE RETURN ST R9,PRELEN SET LENGTH (ZERO AFTER EOF) ST R15,PRECODE L R13,4(,R13) RETURN (14,12),RC=0 RETURN SPACE CLEANUP DS 0H CLI ISOPEN,0 IS INPUT FILE OPEN? BE RETURN NO - JUST RETURN B EOF CLEAN IT UP SPACE5 ******************************************************************** * EOF ENTERED AT END-OF-FILE * ******************************************************************** EOF CLOSE INDATA CLOSE INPUT DATA SET MVI ISOPEN,0 INDICATE FILE NOT OPEN ******************************************************************** NOREC SR R15,R15 SET ZERO RETURN CODE SR R9,R9 SET ZERO LENGTH B RETURN RETURN * BADRET LA R15,16 SET RETURN CODE FOR ERROR SR R9,R9 SET LENGTH = 0 B RETURN ERROR RETURN EJECT * * CONSTANTS * R0 EQU 0 R1 EQU 1 R2 EQU 2 R3 EQU 3 R4 EQU 4 R5 EQU 5 R6 EQU 6 R7 EQU 7 R8 EQU 8 R9 EQU 9 R10 EQU 10 Teradata FastLoad Reference 193 Appendix C: INMOD and Notify Exit Routine Examples z/OS R11 R12 R13 R14 R15 * * * EQU EQU EQU EQU EQU EJECT 11 12 13 14 15 DATA STRUCTURES AND VARIABLES SPACE SAVEAREA DC SAMPNUM DC ISOPEN DC SPACE INDATA DCB PREBUF DSECT PRECODE DS PRELEN DS ROWSIZE EQU PREREC DS DCBD END 18F’0’ SAVE AREA F’0’ XL1’00’ OPEN FILE INDICATOR 10 DDNAME=INDATA,MACRF=(GM),DSORG=PS,EODAD=EOF F F 31774 MAXIMUM ROW SIZE 0XL(ROWSIZE) DEVD=DA,DSORG=PS COBOL INMOD Example The INMOD in the following example reads five 80-byte records from the input data file, combines them, then returns a single, 400-byte record to the Teradata FastLoad utility. IDENTIFICATION DIVISION. PROGRAM-ID. BLKEXIT. AUTHOR. STV . INSTALLATION. TERADATA. DATE-WRITTEN. 05 APRIL 1984. DATE-COMPILED. SECURITY. OPEN. REMARKS. THIS PROGRAM IS AN EXAMPLE OF A COBOL INMOD ROUTINE. THE NAME MUST BE BLKEXIT. FUNCTION: THE PROGRAM READS FIVE 80-BYTE RECORDS AND RETURNS A COMPOSITE RECORD WHICH IS 400 BYTES LONG. ENVIRONMENT DIVISION. CONFIGURATION SECTION. SOURCE-COMPUTER. IBM-370. OBJECT-COMPUTER. IBM-370. INPUT-OUTPUT SECTION. FILE-CONTROL. SELECT INMOD-DATA-FILE ASSIGN TO SYSIN-INDATA. DATA DIVISION. FILE SECTION. FD INMOD-DATA-FILE BLOCK CONTAINS 0 RECORDS LABEL RECORDS STANDARD. 01 INPUT-PARM-AREA PICTURE IS X(80). 194 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples z/OS WORKING-STORAGE SECTION. 01 NUMIN PIC S9(4) COMP VALUE +0. 01 NUMOUT PIC S9(4) COMP VALUE +0. 01 FILES_OPEN PIC S9(4) COMP VALUE +0. LINKAGE SECTION. 01 STRUCT. 02 RETURN-INDICATEPIC S9(9) COMP. 02 RECORD-LEN PIC S9(9) COMP. 02 RECORD-BODY. 03 DATA-AREA1 PIC X(80). 03 DATA-AREA2 PIC X(80). 03 DATA-AREA3 PIC X(80). 03 DATA-AREA4 PIC X(80). 03 DATA-AREA5 PIC X(80). PROCEDURE DIVISION USING STRUCT. BEGIN. MAIN. IF RETURN-INDICATE = 0 THEN DISPLAY ’BLKEXIT CALLED - RETURN CODE 0’ PERFORM OPEN-FILES PERFORM READ-RECORDS GOBACK ELSE IF RETURN-INDICATE = 1 THEN DISPLAY ’BLKEXIT CALLED - RETURN CODE 1’ PERFORM READ-RECORDS GOBACK ELSE IF RETURN-INDICATE = 2 THEN DISPLAY ’BLKEXIT CALLED - RETURN CODE 2’ PERFORM OPEN-FILES MOVE 0 TO RECORD-LEN GOBACK ELSE IF RETURN-INDICATE = 5 THEN DISPLAY ’BLKEXIT CALLED - RETURN CODE 5’ IF FILES_OPEN = 1 THEN CLOSE INMOD-DATA-FILE MOVE 0 TO FILES_OPEN. MOVE 0 TO RECORD-LEN GOBACK ELSE DISPLAY ’BLKEXIT CALLED - RETURN CODE X’ GOBACK. OPEN-FILES SECTION. OPEN INPUT INMOD-DATA-FILE. MOVE 1 TO FILES_OPEN. MOVE 0 TO RETURN-INDICATE. READ-RECORDS SECTION. MOVE 0 TO BLOCK-NUM. READ INMOD-DATA-FILE INTO DATA-AREA1 AT END GO TO END-DATA. ADD 1 TO NUMIN. READ INMOD-DATA-FILE INTO DATA-AREA2 AT END GO TO END-DATA. ADD 1 TO NUMIN. READ INMOD-DATA-FILE INTO DATA-AREA3 AT END GO TO END-DATA. Teradata FastLoad Reference 195 Appendix C: INMOD and Notify Exit Routine Examples z/OS ADD 1 TO NUMIN. READ INMOD-DATA-FILE INTO DATA-AREA4 AT END GO TO END-DATA ADD 1 TO NUMIN. READ INMOD-DATA-FILE INTO DATA-AREA5 AT END GO TO END-DATA. ADD 1 TO NUMIN. MOVE 0 TO RETURN-INDICATE. MOVE 400 TO RECORD-LEN. ADD 1 TO NUMOUT. END-DATA SECTION. MOVE 0 TO RETURN-INDICATE. CLOSE INMOD-DATA-FILE. MOVE 0 TO FILES_OPEN. DISPLAY ’NUMBER OF INPUT RECORDS =’ NUMIN. DISPLAY ’NUMBER OF OUTPUT RECORDS=’ NUMOUT. MOVE 0 TO RECORD-LEN. MOVE 99 TO RETURN-INDICATE. GOBACK. PL/I INMOD Example The INMOD in the following example reads the data from the input file and presents it as read to the Teradata FastLoad utility. This basic program can be modified to select records that meet a specified criteria, convert certain fields, or perform other preprocessing functions. BLKEXIT: PROC (PARM_LIST) OPTIONS(MAIN); /* Routine name must always be BLKEXIT */ DCL EXPORT FILE RECORD INPUT; DCL EOF FIXED BINARY (31,0) INIT (0); DCL THIS_LENGTH FIXED BINARY (31,0); DCL1 PARM_LIST, 10 STATUS FIXED BINARY (31,0), 10 RLENGTH FIXED BINARY (31,0), 10 CCDB_REC CHAR (80); ON ENDFILE (EXPORT) EOF = 1; IF STATUS = 3 /* disregard restarts and checkpoints */ THEN GO TO PROC_RETURN; IF STATUS = 4 /* disregard restarts and checkpoints */ THEN GO TO PROC_RETURN; IF STATUS = 2 /* client restart */ /*Note:STATUS values 2 , 3 and 4 are only applicable to the FastLoad program. An INMOD program can be written to interface with either FastLoad or Bulk Data Load -STATUS values greater than ’1’ will not be set by the Bulk Data Load program. THEN DO; OPEN FILE (EXPORT); GO TO PROC_RETURN; END; IF STATUS = 0 /* CHECK FOR FIRST-TIME-THRU CYCLE */ THEN DO; OPEN FILE (EXPORT); END; RLENGTH = 80; 196 */ Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS READ FILE (EXPORT) INTO (CCDB_REC); IFEOF = 1 THEN DO; CLOSE FILE (EXPORT); STATUS=1; /* INDICATE EOF TO BDL OR FDL */ RLENGTH = 0; GO TO PROC_RETURN; END; STATUS=0; /* INDICATE NEXT RECORD TO BDL OR FDL */ /* Additional processing can be done at this point */ PROC_RETURN: END BLKEXIT ‘OPTIONS(MAIN)’; IBM C INMOD Example Refer to “BLKEXITR.C Sample INMOD” on page 202. The BLKEXITR.C sample, because it is written in C and the entry point is written for all platforms, would work on the mainframe for an IBM C INMOD. UNIX OS BLKEXIT.C Sample INMOD Following is the listing of the BLKEXIT.C sample INMOD routine that is provided with the Teradata FastLoad utility software: /************************************************************************* * * Title: BLKEXIT - sample INMOD routine * * Copyright (C) 1996-2012 by Teradata Corporation. * All Rights Reserved. * Teradata Corporation CONFIDENTIAL AND TRADE SECRET * * This copyrighted material is the Confidential, Unpublished * Property of the Teradata Corporation. This copyright notice and * any other copyright notices included in machine readable * copies must be reproduced on all authorized copies. * * * Description This file contains a sample INMOD, written in C. * This file does not support DBS restarts. * * * History Information * * Revision Date DCR DID Comments * ----------- -------- ----- ------- --------------------------------------* 13.00.00.01 09072007 114414 NT185003 Teradata Corporation Copyright * 13.00.00.00 09012007 100894 SV185048 Visual Studio 8.0 build * 07.07.00.01 06/22/05 96206 CSG Port to HP-UX-11.23 on Itanium * 07.01.00.01 11/12/98 44120 SF3 Release for FastLoad 7.1 * 06.00.00.00 07/18/96 34208 SF3 Release for FastLoad 6.0 * * Teradata FastLoad Reference 197 Appendix C: INMOD and Notify Exit Routine Examples UNIX OS * How to build this INMOD on a UNIX operating system: * * Compile and link it into a shared object: * * cc -G -KPIC <inmod-name>.c -o <shared-object-name> * * * How to use this program: * * This INMOD routine will generate 2 columns of data: * * 4-byte integer counter for the Unique Primary Index * 10-byte character string * * This sample INMOD will generate 100,000 rows, if no RECORD * statement is used in the FastLoad job script. * * A sample of the job script for this INMOD would be as follows: * * * use your own system, account and password here. * LOGON tdpid/user,password; DROP TABLE Error_1; DROP TABLE Error_2; DROP TABLE TestTable; CREATE TABLE TestTable AS ( Counter Integer, text char(10) ) UNIQUE PRIMARY INDEX(Counter); BEGIN LOADING TestTable ErrorFiles Error_1, Error_2; DEFINE Counter (Integer), text (char(10)) INMOD=<shared-object-name>; INSERT INTO TestTable (Counter, text) VALUES (:Counter, :text); END LOADING; LOGOFF; *************************************************************************/ #include <stdio.h> #include <string.h> #define FILEOF #define EM_OK 401 0 #define NUMROWS #define ROWSIZE 100000 64000 198 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS typedef int Int32; /* */ typedef unsigned int UInt32; /* */ typedef short Int16; typedef struct inmod_struct { Int32 ReturnCode; Int32 Length; char Body[ROWSIZE]; } inmdtyp,*inmdptr; inmdptr inmodptr; char *str = "123test890"; Int32 reccnt = 0; /************************************************************************* * * MakeRecord - Generate a record * * This module creates the data record * * In this example, we are just generating dummy records * with canned data, only we change the first column, * essentially a record number, so that each row is unique. * * For performance reasons, we do not change the 2nd column. * *************************************************************************/ Int32 MakeRecord() { char *p; /* have we reached EOF yet? */ if (reccnt >= NUMROWS) return(FILEOF); /* nope. get start of buffer */ p = inmodptr->Body; /* place column 1, a unique primary index */ memcpy(p, &reccnt, (UInt32)sizeof(reccnt)); p += sizeof(reccnt); /* place column 2, a string */ memcpy(p, str, strlen(str)); p += strlen(str); inmodptr->ReturnCode = 0; inmodptr->Length = p - inmodptr->Body; reccnt++; return(EM_OK); } Teradata FastLoad Reference 199 Appendix C: INMOD and Notify Exit Routine Examples UNIX OS /************************************************************************* * * HostRestart - Host restarted * * Reset and start sending data from the begining. * *************************************************************************/ Int32 HostRestart() { return(EM_OK); } /************************************************************************* * * CheckPoint - Save checkpoint * *************************************************************************/ Int32 CheckPoint() { return(EM_OK); } /************************************************************************* * * DBSRestart - DBS restarted * * Do what you have to do. * Reset record counter to checkpoint value * *************************************************************************/ Int32 DBSRestart() { return(EM_OK); } /************************************************************************* * * CleanUp - Do cleanup. * *************************************************************************/ Int32 CleanUp() { return(EM_OK); } /************************************************************************* * * InvalidCode - Invalid INMOD code returned. * *************************************************************************/ Int32 InvalidCode() { fprintf(stderr, "**** Invalid code received by INMOD\n"); return(EM_OK); } /************************************************************************* * 200 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS * Init - initialize some stuff * * Do any initialization necessary * *************************************************************************/ Int32 Init() { return(EM_OK); } /************************************************************************* * * BLKEXIT - Start processing * * This is the main module which contains the checks for * number of records generated and buffer filling. This * module also sends the filled buffer to the DBS. * *************************************************************************/ #if defined WIN32 __declspec(dllexport) Int32 BLKEXIT(tblptr) #elif defined I370 Int32 _dynamn(tblptr) #else Int32 BLKEXIT(tblptr) #endif char *tblptr; { Int32 result; inmodptr = (struct inmod_struct *)tblptr; /* process the function passed to the INMOD */ switch (inmodptr->ReturnCode) { case 0: result = Init(); if (result) break; result = MakeRecord(); break; case 1: result = MakeRecord(); break; case 2: result = HostRestart(); break; case 3: result = CheckPoint(); break; case 4: result = DBSRestart(); break; case 5: result = CleanUp(); break; default: result = InvalidCode(); } /* see if we have reached EOF condition */ if (result == FILEOF) { inmodptr->Length = 0; result = EM_OK; } Teradata FastLoad Reference 201 Appendix C: INMOD and Notify Exit Routine Examples UNIX OS return(result); } BLKEXITR.C Sample INMOD Following is the listing of the BLKEXITR.C sample INMOD routine that is provided with the Teradata FastLoad utility software: /************************************************************************* * * Title: BLKEXITR - sample INMOD routine * * Copyright (C) 1996-2012 by Teradata Corporation. * All Rights Reserved. * Teradata Corporation CONFIDENTIAL AND TRADE SECRET * * This copyrighted material is the Confidential, Unpublished * Property of the Teradata Corporation. This copyright notice and * any other copyright notices included in machine readable * copies must be reproduced on all authorized copies. * * * Description This file contains a sample INMOD, written in C. * This file supports DBS restarts. * * * History Information * * Revision Date DCR DID Comments * ----------- -------- ----- ------- --------------------------------------* 13.00.00.01 09072007 114414 NT185003 Teradata Corporation Copyright * 13.00.00.00 09012007 100894 SV185048 Visual Studio 8.0 build * 07.07.00.01 06/23/05 96206 CSG Port to HPUX-11.23 on Itanium * 07.01.00.01 11/12/98 44120 SF3 Release for FastLoad 7.1 * 06.00.00.00 07/18/96 34208 SF3 Release for FastLoad 6.0 * * * How to build this INMOD on a UNIX operating system: * * Compile and link it into a shared object: * * cc -G -KPIC <inmod-name>.c -o <shared-object-name> * * * How to use this program: * * This INMOD routine will generate 2 columns of data: * * 4-byte integer counter for the Unique Primary Index * 10-byte character string * * This sample INMOD will generate 100,000 rows, if no RECORD * statement is used in the FastLoad job script. * * A sample of the job script for this INMOD would be as follows: * * 202 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS * use your own system, account and password here. * LOGON tdpid/user,password; DROP TABLE Error_1; DROP TABLE Error_2; DROP TABLE TestTable; CREATE TABLE TestTable AS ( Counter Integer, text char(10) ) UNIQUE PRIMARY INDEX(Counter); BEGIN LOADING TestTable ErrorFiles Error_1, Error_2; DEFINE Counter (Integer), text (char(10)) INMOD=<shared-object-name>; INSERT INTO TestTable (Counter, text) VALUES (:Counter, :text); END LOADING; LOGOFF; *************************************************************************/ #include <stdio.h> #include <string.h> #define FILEOF #define EM_OK 401 0 #define NUMROWS #define ROWSIZE 100000 64000 typedef int Int32; /* */ typedef unsigned int UInt32; /* */ typedef short Int16; typedef struct inmod_struct { Int32 ReturnCode; Int32 Length; char Body[ROWSIZE]; } inmdtyp,*inmdptr; inmdptr inmodptr; char *str = "123test890"; char *fname = "chkpoint.dat"; FILE *fp = NULL; Int32 reccnt = 0; Int32 chkpnt; Teradata FastLoad Reference 203 Appendix C: INMOD and Notify Exit Routine Examples UNIX OS /************************************************************************* * * MakeRecord - Generate a record * * This module creates the data record * * In this example, we are just generating dummy records * with canned data, only we change the first column, * essentially a record number, so that each row is unique. * *************************************************************************/ Int32 MakeRecord() { char *p; /* have we reached EOF yet? */ if (reccnt >= NUMROWS) return(FILEOF); /* nope. get start of buffer */ p = inmodptr->Body; /* place column 1, a unique primary index */ memcpy(p, &reccnt, (UInt32)sizeof(reccnt)); p += sizeof(reccnt); /* place column 2, a string */ memcpy(p, str, strlen(str)); p += strlen(str); inmodptr->ReturnCode = 0; inmodptr->Length = p - inmodptr->Body; reccnt++; return(EM_OK); } /************************************************************************* * * HostRestart - Host restarted * * Retrieve the checkpoint information from the checkpoint file. * Reset record counter to checkpoint value * *************************************************************************/ Int32 HostRestart() { Int32 result; /* see if the file is already open */ if (!fp) { fp = fopen(fname, "r+"); if (!fp) 204 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS return(!EM_OK); } rewind(fp); result = fread(&chkpnt, sizeof(chkpnt), 1, fp); if (result != 1) { fprintf(stderr, "INMOD: ERROR READING CHECKPOINT FILE\n"); fprintf(stderr, "INMOD: %d ELEMENTS WERE READ\n", result); perror("INMOD"); return(!EM_OK); } fprintf(stderr, "INMOD: HOST RESTARTED. CHECKPOINT: %d\n", chkpnt); reccnt = chkpnt; return(EM_OK); } /************************************************************************* * * CheckPoint - Save checkpoint * *************************************************************************/ Int32 CheckPoint() { Int32 result; chkpnt = reccnt; rewind(fp); result = fwrite(&chkpnt, sizeof(chkpnt), 1, fp); if (result != 1) { fprintf(stderr, "INMOD: ERROR WRITING TO CHECKPOINT FILE\n"); fprintf(stderr, "INMOD: %d ELEMENTS WERE WRITTEN\n", result); perror("INMOD"); return(!EM_OK); } fprintf(stderr, "INMOD: CHECKPOINT AT ROW: %d\n", chkpnt); return(EM_OK); } /************************************************************************* * * DBSRestart - DBS restarted * * Retrieve the checkpoint information from the checkpoint file. * Reset record counter to checkpoint value * *************************************************************************/ Int32 DBSRestart() { Int32 result; /* see if the file is already open */ if (!fp) { Teradata FastLoad Reference 205 Appendix C: INMOD and Notify Exit Routine Examples UNIX OS fp = fopen(fname, "r+"); if (!fp) return(!EM_OK); } rewind(fp); result = fread(&chkpnt, sizeof(chkpnt), 1, fp); if (result != 1) { fprintf(stderr, "INMOD: ERROR READING CHECKPOINT FILE\n"); fprintf(stderr, "INMOD: %d ELEMENTS WERE READ\n", result); perror("INMOD"); return(!EM_OK); } fprintf(stderr, "INMOD: DBS RESTARTED. CHECKPOINT: %d\n", chkpnt); reccnt = chkpnt; return(EM_OK); } /************************************************************************* * * CleanUp - Do cleanup. * * Here we close the file and then remove it. * *************************************************************************/ Int32 CleanUp() { fclose(fp); remove(fname); return(EM_OK); } /************************************************************************* * * InvalidCode - Invalid INMOD code returned. * *************************************************************************/ Int32 InvalidCode() { fprintf(stderr, "**** Invalid code received by INMOD\n"); return(EM_OK); } /************************************************************************* * * Init - initialize some stuff * * Do any initialization necessary * * For this example, we will open a disk file to hold * the checkpoint information. For this example, we * will just store the row number. If this INMOD was * reading data from a file, then the file position * would need to be stored. * *************************************************************************/ 206 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples UNIX OS Int32 Init() { fp = fopen(fname, "w"); if (!fp) return(!EM_OK); return(EM_OK); } /************************************************************************* * * BLKEXIT - Start processing * * This is the main module which contains the checks for * number of records generated and buffer filling. This * module also sends the filled buffer to the DBS. * *************************************************************************/ #if defined WIN32 __declspec(dllexport) Int32 BLKEXIT(tblptr) #elif defined I370 Int32 _dynamn(tblptr) #else Int32 BLKEXIT(tblptr) #endif char *tblptr; { Int32 result; inmodptr = (struct inmod_struct *)tblptr; /* process the function passed to the INMOD */ switch (inmodptr->ReturnCode) { case 0: result = Init(); if (result) break; result = MakeRecord(); break; case 1: result = MakeRecord(); break; case 2: result = HostRestart(); break; case 3: result = CheckPoint(); break; case 4: result = DBSRestart(); break; case 5: result = CleanUp(); break; default: result = InvalidCode(); } /* see if we have reached EOF condition */ if (result == FILEOF) { inmodptr->Length = 0; result = EM_OK; } Teradata FastLoad Reference 207 Appendix C: INMOD and Notify Exit Routine Examples All Platforms return(result); } All Platforms Notify Exit Routine Example The user exit routine in the following example processes events specified by the NOTIFY command. =============================================================================== /*************************************************************************** * * TITLE: flnfyext.c .... an example notify exit... * * Copyright(C) 1996-2005, 2007-2012 by Teradata Corporation. * All Rights Reserved. * Teradata Corporation CONFIDENTIAL AND TRADE SECRET * * This copyrighted material is the Confidential, Unpublished * Property of the Teradata Corporation. This copyright notice and * any other copyright notices included in machine readable * copies must be reproduced on all authorized copies. * * * Description This file is a sample user exit routine for * processing NOTIFY events * * History Information * * Revision Date DCR DID Comments * ----------- -------- ----- -------- -------------------------------------* 14.10.00.00 10192012 SA-28595 NT185003 FL supports Notify Exit EON * 14.00.00.04 06292012 SA-1863 NT185003 Support 8-byte row count (DR141961) * 14.00.00.03 03112011 117582 AS185203 EON Support * 14.00.00.02 11152010 141961 NT185003 Support 8 byte row count * 14.00.00.01 11032010 109902 SV185048 Added database name to notify * 13.00.00.01 09072007 114414 NT185003 Teradata Corporation Copyright * 13.00.00.00 09012007 100894 SV185048 Visual Studio 8.0 build * 07.07.00.01 09292005 96206 CSG Port to HPUX-IA64 platform * 07.06.00.02 06182004 68859 XX151000 Back out branding changes * 07.06.00.01 01142004 68859 XX151000 Change DBS to TERADATA DATABASE * 07.01.00.01 07/10/98 42517 SF3 Redesign User Exit Interface * 06.01.00.01 04/04/97 34579 SF3 Initial Version * * * Notes The entry point to this User Exit must * be called "_dynamn" * Notes DR141961 and SA-1863 : support 8 byte row counters * To display correctly, please use the events: * * NFEventCheckPoint64 = 13 * NFEventPhaseIEnd64 = 14 * NFEventPhaseIIEnd64 = 15 * NFEventDropErrTableI64 = 16 * NFEventDropErrTableII64= 17 208 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples All Platforms * * instead of * * NFEventCheckPoint = 3 * NFEventPhaseIEnd = 4 * NFEventPhaseIIEnd = 6 * NFEventDropErrTableI = 7 * NFEventDropErrTableII = 8 * * Note SA-28595, please use 2 new events * * NFEventInitializeEON = 18 * NFEventPhaseIBeginEON = 19 * * * FastLoad works with both OLD and NEW Notify Exit. * ***************************************************************************/ #include <stdio.h> /* DR 96206 --> */ typedef int Int32; typedef unsigned int UInt32; #ifdef WIN32 typedef unsigned __int64 UInt64 ; #else typedef unsigned long long UInt64 ; #endif /* DR 96206 <-- */ typedef enum { NFEventInitialize = NFEventFileInmodOpen = NFEventPhaseIBegin = NFEventCheckPoint = NFEventPhaseIEnd = NFEventPhaseIIBegin = NFEventPhaseIIEnd = NFEventDropErrTableI = NFEventDropErrTableII = NFEventDBSRestart = NFEventCLIError = NFEventDBSError = NFEventExit = /* DR141961 ==> */ NFEventCheckPoint64 = NFEventPhaseIEnd64 = NFEventPhaseIIEnd64 = NFEventDropErrTableI64 = NFEventDropErrTableII64= /* DR141961 <== */ /* SA-28595 ==> */ NFEventInitializeEON = NFEventPhaseIBeginEON = /* SA-28595 <== */ } NfyFLDEvent; #define ID_FASTLOAD #define ID_MULTILOAD #define ID_FASTEXPORT Teradata FastLoad Reference /* DR141961 */ /* DR141961 */ 0 1 2 3 4 5 6 7 8 9 10 11 12 , , , , , , , , , , , , , 13 14 15 16 17 , , , , , 18 , 19 1 2 3 209 Appendix C: INMOD and Notify Exit Routine Examples All Platforms #define ID_BTEQ #define ID_TPUMP #define #define #define #define #define #define #define #define 4 5 MAXVERSIONIDLEN MAXUTILITYNAMELEN MAXUSERNAMELEN MAXUSERSTRLEN MAXTABLENAMELEN MAXFILENAMELEN MAXDBASENAMELEN MAXUINT64 32 32 129 80 129 256 129 24 /* DR117582 */ /* SA-28595 */ /* DR117582 */ /* SA-28595 */ /* DR117582 */ /* DR109902 */ /* SA-28595 */ /* DR141961 */ typedef struct _FLNotifyExitParm { Int32 Event; union { struct { UInt32 VersionLen; char VersionId[MAXVERSIONIDLEN]; UInt32 UtilityId; UInt32 UtilityNameLen; char UtilityName[MAXUTILITYNAMELEN]; UInt32 UserNameLen; char UserName[MAXUSERNAMELEN]; UInt32 UserStringLen; char UserString[MAXUSERSTRLEN]; } Initialize; /* SA-28595 ==> */ struct { UInt32 VersionLen; char VersionId[MAXVERSIONIDLEN]; UInt32 UtilityId; UInt32 UtilityNameLen; char UtilityName[MAXUTILITYNAMELEN]; UInt32 UserNameLen; char *UserName; UInt32 UserStringLen; char UserString[MAXUSERSTRLEN]; } InitializeEON; /* SA-28595 <== */ struct { UInt32 FileNameLen; char FileOrInmodName[MAXFILENAMELEN]; } FileInmodOpen ; struct { char DBaseName[MAXDBASENAMELEN]; /* DR109902 */ UInt32 TableNameLen; char TableName[MAXTABLENAMELEN]; UInt32 dummy; } PhaseIBegin ; /* SA-28595 ==> */ struct { char *DBaseName; UInt32 TableNameLen; char *TableName; UInt32 dummy; } PhaseIBeginEON ; /* SA-28595 <== */ struct { UInt32 RecordCount; 210 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples All Platforms } CheckPoint; /* DR141961 ==> */ struct { char RecordCount[MAXUINT64]; } CheckPoint64; /* DR141961 <== */ struct { UInt32 RecsRead; UInt32 RecsSkipped; UInt32 RecsRejected; UInt32 RecsSent; } PhaseIEnd ; /* DR141961 ==> */ struct { char RecsRead[MAXUINT64]; char RecsSkipped[MAXUINT64]; char RecsRejected[MAXUINT64]; char RecsSent[MAXUINT64]; } PhaseIEnd64 ; /* DR141961 <== */ struct { UInt32 dummy; } PhaseIIBegin; struct { UInt32 Inserts; UInt32 dummy1; UInt32 dummy2; UInt32 dummy3; } PhaseIIEnd; /* DR141961 ==> */ struct { char Inserts[MAXUINT64]; UInt32 dummy1; UInt32 dummy2; UInt32 dummy3; } PhaseIIEnd64; /* DR141961 <== */ struct { UInt32 Rows; UInt32 dummy; } DropErrTableI; /* DR141961 ==> */ struct { char Rows[MAXUINT64]; UInt32 dummy; } DropErrTableI64; /* DR141961 <== */ struct { UInt32 Rows; UInt32 dummy; } DropErrTableII ; /* DR141961 ==> */ struct { char Rows[MAXUINT64]; UInt32 dummy; } DropErrTableII64 ; /* DR141961 <== */ struct { UInt32 dummy; Teradata FastLoad Reference 211 Appendix C: INMOD and Notify Exit Routine Examples All Platforms } DBSRestart; struct { UInt32 ErrorCode; } CLIError; struct { UInt32 ErrorCode; } DBSError; struct { UInt32 ReturnCode; } Exit; } Vals; } FLNotifyExitParm; /************************************************************************* * * CODE STARTS HERE * *************************************************************************/ #ifdef WIN32 __declspec(dllexport) Int32 _dynamn(FLNotifyExitParm *P) #else Int32 _dynamn(FLNotifyExitParm *P) #endif { static FILE *fp; if (!fp) { #ifdef I370 if (!(fp = fopen("NFYEXIT", "w"))) return(1); #else if (!(fp = fopen("NFYEXIT.OUT", "w"))) return(1); #endif } switch(P->Event) { case NFEventInitialize : fprintf(fp, "exit called @ FastLoad Notify initialization.\n"); fprintf(fp, " Version Id: %s.\n", P->Vals.Initialize.VersionId); fprintf(fp, " Utility Id: %d.\n", P->Vals.Initialize.UtilityId); fprintf(fp, " Utility Name: %s.\n", P->Vals.Initialize.UtilityName); fprintf(fp, " User Name: %s.\n", P->Vals.Initialize.UserName); break; /* SA-28595 ==> */ case NFEventInitializeEON : fprintf(fp, "exit called @ FastLoad Notify initialization.\n"); fprintf(fp, 212 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples All Platforms " Version Id: %s.\n", P->Vals.InitializeEON.VersionId); fprintf(fp, " Utility Id: %d.\n", P->Vals.InitializeEON.UtilityId); fprintf(fp, " Utility Name: %s.\n", P->Vals.InitializeEON.UtilityName); fprintf(fp, " User Name: %s.\n", P->Vals.InitializeEON.UserName); break; /* SA-28595 <== */ case NFEventFileInmodOpen : fprintf(fp, "exit called @ FastLoad file/inmod open: %s.\n", P->Vals.FileInmodOpen.FileOrInmodName); break; case NFEventPhaseIBegin : fprintf(fp, "exit called @ FastLoad phase I (start) for Database %s.\n", P->Vals.PhaseIBegin.DBaseName); /* 109902 */ fprintf(fp, "exit called @ FastLoad phase I (start) for table %s.\n", P->Vals.PhaseIBegin.TableName); break; /* SA-28595 ==> */ case NFEventPhaseIBeginEON : fprintf(fp, "exit called @ FastLoad phase I (start) for Database %s.\n", P->Vals.PhaseIBeginEON.DBaseName); /* 109902 */ fprintf(fp, "exit called @ FastLoad phase I (start) for table %s.\n", P->Vals.PhaseIBeginEON.TableName); break; /* SA-28595 <== */ case NFEventCheckPoint : fprintf(fp, "exit called @ FastLoad checkpoint : %lu records loaded.\n", P->Vals.CheckPoint.RecordCount); break; /* DR141961 ==> */ case NFEventCheckPoint64 : fprintf(fp, "exit called @ FastLoad checkpoint : %s records loaded.\n", P->Vals.CheckPoint64.RecordCount); break; /* DR141961 <== */ case NFEventPhaseIEnd : fprintf(fp, "exit called @ FastLoad phase I (end).\n"); fprintf(fp, " Records read: %lu\n", Teradata FastLoad Reference 213 Appendix C: INMOD and Notify Exit Routine Examples All Platforms P->Vals.PhaseIEnd.RecsRead); fprintf(fp, " Records skipped: %lu\n", P->Vals.PhaseIEnd.RecsSkipped); fprintf(fp, " Records rejected: %lu\n", P->Vals.PhaseIEnd.RecsRejected); fprintf(fp, " Records sent: %lu\n", P->Vals.PhaseIEnd.RecsSent); break; /* DR141961 ==> */ case NFEventPhaseIEnd64 : fprintf(fp, "exit called @ FastLoad phase I (end).\n"); fprintf(fp, " Records read: %s\n", P->Vals.PhaseIEnd64.RecsRead); fprintf(fp, " Records skipped: %s\n", P->Vals.PhaseIEnd64.RecsSkipped); fprintf(fp, " Records rejected: %s\n", P->Vals.PhaseIEnd64.RecsRejected); fprintf(fp, " Records sent: %s\n", P->Vals.PhaseIEnd64.RecsSent); break; /* DR141961 <== */ case NFEventPhaseIIBegin : fprintf(fp, "exit called @ FastLoad phase II (start).\n"); break; case NFEventPhaseIIEnd : fprintf(fp, "exit called @ FastLoad phase II (end): %lu records loaded.\n", P->Vals.PhaseIIEnd.Inserts); break; /* DR141961 ==> */ case NFEventPhaseIIEnd64 : fprintf(fp, "exit called @ FastLoad phase II (end): %s records loaded.\n", P->Vals.PhaseIIEnd64.Inserts); break; /* DR141961 <== */ case NFEventDropErrTableI : fprintf(fp, "exit called @ FastLoad ET 1 Drop : %lu records in table.\n", P->Vals.DropErrTableI.Rows); break; /* DR141961 ==> */ case NFEventDropErrTableI64 : fprintf(fp, 214 Teradata FastLoad Reference Appendix C: INMOD and Notify Exit Routine Examples All Platforms "exit called @ FastLoad ET 1 Drop : %s records in table.\n", P->Vals.DropErrTableI64.Rows); break; /* DR141961 <== */ case NFEventDropErrTableII : fprintf(fp, "exit called @ FastLoad ET 2 Drop : %lu records in table.\n", P->Vals.DropErrTableII.Rows); break; /* DR141961 ==> */ case NFEventDropErrTableII64 : fprintf(fp, "exit called @ FastLoad ET 2 Drop : %s records in table.\n", P->Vals.DropErrTableII64.Rows); break; /* DR141961 <== */ case NFEventDBSRestart : fprintf(fp, "exit called @ FastLoad detects DBS restart.\n"); break; case NFEventCLIError : fprintf(fp, "exit called @ FastLoad detects CLI error: %d.\n", P->Vals.CLIError.ErrorCode); break; case NFEventDBSError : fprintf(fp, "exit called @ FastLoad detects DBS error: %d.\n", P->Vals.DBSError.ErrorCode); break; case NFEventExit : fprintf(fp, "exit called @ FastLoad exiting. Return code: %d.\n", P->Vals.Exit.ReturnCode); fclose(fp); break; } return(0); } /****************************************************************************** * * End of FlNfyExt.c * ******************************************************************************/ Teradata FastLoad Reference 215 Appendix C: INMOD and Notify Exit Routine Examples All Platforms 216 Teradata FastLoad Reference APPENDIX D Compile, Link, and Execute INMOD and Notify Exit Routines This appendix provides examples of compiling, linking, and executing INMOD and notify exit routines written in the supported programming languages for the following client system platforms. • z/OS • IBM C or C++ INMODs • UNIX OS • Windows Listings are provided of sample INMOD routines and notify exit routines written in: • Assembler • COBOL • C or C++ • IBM C • PL/I Mainframe-attached z/OS client systems support INMOD and notify exit routines written in any of the languages listed. Network-attached UNIX and Windows client systems support routines written in C. z/OS Assembler INMOD routines written in Assembler do not need to be link-edited with other load modules. To create and use an Assembler INMOD routine on z/OS 1 Prepare the INMOD routine source file. 2 Compile the INMOD program. 3 Invoke the Teradata FastLoad utility to execute the INMOD routine. Teradata FastLoad Reference 217 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS Example The INMOD routine in the following example reads a record from the input data source and adds a four--byte integer field to the front of the record. The new field contains a sequence record that ranges from one to the total number of input records. //JLHFAST JOB 1,’FASTLOAD/INMOD’,MSGCLASS=A,CLASS=B,NOTIFY=JLH // ****************************************************************** //* Compile and link-edit the INMOD * //**************************************************************** //ASMCOMPL EXEC ASMFCL //**************************************************************** //* The INMOD starts here * //**************************************************************** //ASM.SYSIN DD * <Assembler source goes here> /* //LKED.SYSLMOD DD DSN=JLH.INMOD.LOAD,DISP=SHR //LKED.SYSIN DD * NAME ASMINMOD(R) /* //**************************************************************** //* Execute BTEQ to create a Teradata DBS table * //* * //* The BTEQ statement .run ddname=logon allows the user to * //* place sensitive logon information in a secure data set (in * //* this case, the ddname is logon). Logging on from a data * //* set prevents this sensitive information from appearing in * //* the SYSIN file. * //**************************************************************** /* //BTEQ EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon drop table asm_example; drop table asm_error1; drop table asm_error2; create table asm_example (f1 INTEGER, f2 CHAR(80)) primary index(f1); .quit ## //* //****************************************************************** //* Execute FastLoad * //****************************************************************** //FAST EXEC TDSFAST //FASTLOAD.STEPLIB DD // DD DSN=JLH.INMOD.LOAD,DISP=SHR //FASTLOAD.SYSIN DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR // DD DATA,DLM=$$ BEGIN LOADING asm_example ERRORFILES asm_error1,asm_error2; DEFINE T1 (integer), T2 (CHAR(80)) INMOD=ASMINMOD; INSERT asm_example (:t1,:t2); END LOADING; LOGOFF; $$ 218 Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS //FASTLOAD.INDATA DDDSN=JLH.ASMXMPL.DATA,DISP=SHR //**************************************************************** //* Using BTEQ, select the rows loaded by FastLoad * //**************************************************************** //BTEQ EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .runddname logon select * from asm_example order by 1; .quit ## COBOL INMOD routines written in COBOL on z/OS must: • Contain the entry point name BLKEXIT • Be link-edited with the TERCOBOL load module. • Be stored as an z/OS load library file To create and use a COBOL INMOD routine on z/OS 1 Prepare the INMOD routine source file. 2 Compile the INMOD routine. 3 Link-edit the INMOD program with the specified load module and store the results as an z/OS load library file. 4 Invoke the Teradata FastLoad utility to execute the INMOD routine. Example The INMOD in the following example reads five 80-byte records from the input data source and returns a single, 400-byte record to Teradata FastLoad. BTEQ1 creates a table on the Teradata Database and BTEQ2 selects rows from the table to check for properly loaded data. //JLHFAST JOB 1,’FASTLOAD/INMOD’,MSGCLASS=A,CLASS=B,NOTIFY=JLH //**************************************************************** //* Compile and link-edit the INMOD * //**************************************************************** //COBCOMPL EXEC COBUCL //**************************************************************** //* The INMOD starts here * //**************************************************************** //COB.SYSIN DD * <COBOL source goes here> //**************************************************************** //* Link-edit the INMOD with the Teradata-supplied module * //* (TERCOBOL) * //**************************************************************** /* //LKED.SYSLIB DD DSN=SYS1.COBLIB,DISP=SHR // DD DSN=TERADATA.APPLOAD,DISP=SHR Teradata FastLoad Reference 219 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS //LKED.SYSLMOD DD DSN=JLH.INMOD.LOAD,DISP=SHR //LKED.SYSIN DD * INCLUDE SYSLIB(TERCOBOL) ENTRY TERCOBOL NAME COBINMOD(R) /* //**************************************************************** //* Execute BTEQ to create a Teradata DBS table * //* * //* The BTEQ statement .run ddname=logon allows the user to * //* place sensitive logon information in a secure data set (in * //* this case, the ddname is logon). Logging on from a data * //* set prevents this sensitive information from appearing in * //* the SYSIN file. * //**************************************************************** //BTEQ1 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon drop table cobol_example; drop table example_error1; drop table example_error2; create table cobol_example (f1 char(10), f2 varchar(390)) primary index(f1); .quit ## //* //FAST EXEC TDSFAST //FASTLOAD.STEPLIB DD // DD DSN=JLH.INMOD.LOAD,DISP=SHR //FASTLOAD.SYSIN DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR // DD DATA,DLM=$$ BEGIN LOADING cobol_example, ERRORFILES cobol_error1,cobol_error2; DEFINE T1 (CHAR(10)) ,T2 (CHAR(390)) INMOD=COBINMOD; INSERT cobol_example (:t1,:t2); END LOADING; LOGOFF; $$ //FASTLOAD.INDATA DD DSN=JLH.COBXMPL.DATA,DISP=SHR //* //**************************************************************** //* Using BTEQ, select the rows loaded by FastLoad * //**************************************************************** //* //BTEQ2 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon select * from cobol_example order by 1; .quit ## IBM C INMOD and notify exit routines written in IBM C must: • 220 Contain the entry point name _dynamn Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS • Be stored as an z/OS load library file To create and use an IBM C INMOD or notify exit routine on z/OS 1 Prepare the INMOD routine source file. 2 Compile the INMOD routine. 3 Link-edit the INMOD program and store the results as an z/OS load library file. 4 Invoke the Teradata FastLoad utility to execute the INMOD routine. Example The INMOD routine in the following example reads the data from the input source and presents it, as read, to the Teradata FastLoad utility. BTEQ1 creates a table on the Teradata Database and BTEQ2 selects rows from the table to check for properly loaded data. This basic program can be modified to select records that meet a specified criteria, convert certain fields, or perform other preprocessing functions. //* Compile and link-edit the INMOD //****************************************************************** //IBMCOMPL EXEC LC370CL,ENTRY=DYNNR,PARM.C='DYNAMNDEF,OPTIMIZE' //C.SYSLIB DD // DD DISP=SHR,DSN=<Your macro library> //C.SYSIN DD * //****************************************************************** //* Link-edit the INMOD * //****************************************************************** //LKED.SYSLMOD DD DISP=SHR,DSN=<Your load library> //LKED.SYSIN DD * NAME MYINMOD(R) //**************************************************************** //* Execute BTEQ to create a Teradata DBS table * //* * //* The BTEQ statement .run ddname=logon allows the user to * //* place sensitive logon information in a secure data set (in * //* this case, the ddname is logon). Logging on from a data * //* set prevents this sensitive information from appearing in * //*the SYSIN file. * //**************************************************************** //BTEQ1 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon drop table stuff; drop table stuff_error1; drop table stuff_error2; create table stuff (t1 char(40), t2 char(40)) primary index(t1); .quit ## //* Teradata FastLoad Reference 221 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS //* //FAST EXEC TDSFAST //FASTLOAD.STEPLIB DD // DD // DD DISP=SHR,DSN=<your load library> //FASTLOAD.SYSIN DD DATA,DLM=$$ LOGON <Logon userid and password> BEGIN LOADING stuff ERRORFILES stuff_error1,stuff_error2; DEFINE t1 (char(40)), t2 (char(40)) INMOD=MYINMOD; INSERT INTO stuff (:t1,:t2); END LOADING; logoff; $$ //FASTLOAD.EXPORT DD DSN=JLH.STUFF.DATA,DISP=SHR //* //**************************************************************** //* Using BTEQ, select the rows loaded by FastLoad * //**************************************************************** //BTEQ2 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon select * from stuff order by 1; .quit PL/I INMOD routines written in PL/I must: • Contain the entry point name DYNAMN • Be link-edited with the PLIA load module • Be stored as an z/OS load library file • Include ‘OPTIONS(MAIN)’ To create and use a PL/I INMOD routine on z/OS: 1 Prepare the INMOD routine source file. 2 Compile the INMOD routine. 3 Link-edit the INMOD program with the specified load module and store the results as an z/OS load library file. 4 Invoke the Teradata FastLoad utility to execute the INMOD routine. Example The INMOD routine in the following example reads the data from the input source and presents it, as read, to the Teradata FastLoad utility. BTEQ1 creates a table on the Teradata Database and BTEQ2 selects rows from the table to check for properly loaded data. 222 Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines z/OS This basic program can be modified to select records that meet a specified criteria, convert certain fields, or perform other preprocessing functions. //IEL1CL EXEC IEL1CL,MEM=MYINMOD //PLI.SYSIN DD DSN=JLH.SOURCE(&MEM),DISP=SHR //PLI.SYSPRINT DD SYSOUT=* //LKED.SYSPRINT DD SYSOUT=* //LKED.SYSLMOD DD DSN=JLH.INMOD.LOAD(&MEM),DISP=SHR //* //LINK EXEC PGM=IEWL,PARM=(LIST,LET,MAP,NORENT,NOREUS) //SYSPRINT DD SYSOUT=* //SYSLOUT DD SYSOUT=* //SYSUT1 DD UNIT=VIO,SPACE=(CYL,(1,1)) //SYSLIB DD DSN=TERADATA.APPLOAD,DISP=SHR // DD DSN=SYS1.SCEELKED,DISP=SHR // DD DSN=JLH.INMOD.LOAD,DISP=SHR //OBJLIB DD DSN=JLH.UTILS.OBJLIB,DISP=SHR //SYSLMOD DD DSN=JLH.INMOD.LOAD,DISP=SHR //SYSLIN DD * INCLUDE SYSLIB(MYINMOD) INCLUDE OBJLIB(PLIA) ENTRY DYNAMN NAME MYINMOD(R) //**************************************************************** //* Execute BTEQ to create a Teradata DBS table * //* * //* The BTEQ statement .run ddname=logon allows the user to * //* place sensitive logon information in a secure data set (in * //* this case, the ddname is logon). Logging on from a data * //* set prevents this sensitive information from appearing in * //* the SYSIN file. * //**************************************************************** //BTEQ1 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon drop table stuff; drop table stuff_error1; drop table stuff_error2; create table stuff (t1 char(40), t2 char(40)) primary index(t1); .quit ## //* //* //FAST EXEC TDSCFAST //FASTLOAD.STEPLIBDD // DD DSN=JLH.INMOD.LOAD,DISP=SHR //FASTLOAD.SYSIN DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR // DD DATA,DLM=$$ BEGIN LOADING stuff ERRORFILES stuff_error1,stuff_error2; DEFINE t1 (char(40)), t2 (char(40)) INMOD=MYINMOD; INSERT INTO stuff (:t1,:t2); END LOADING; logoff; Teradata FastLoad Reference 223 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines IBM C or C++ INMODs $$ //FASTLOAD.EXPORT DD DSN=JLH.STUFF.DATA,DISP=SHR //**************************************************************** //* Using BTEQ, select the rows loaded by FastLoad * //**************************************************************** //BTEQ2 EXEC TDSBTEQ //LOGON DD DSN=JLH.CNTL(JLHLOGON),DISP=SHR //SYSIN DD DATA,DLM=## .run ddname=logon select * from stuff order by 1; .quit IBM C or C++ INMODs Inmods can be written using the various flavors of IBM C or C++ (for example, C/370, AD/ Cycle, OS/390 C/C++, and so on). Teradata FastLoad provides source code for an assembler language stub that invokes the appropriate CEEPIPI calls to support pre-initialization, repeatable invocation, and termination. The procedure follows: 1 Modify the supplied sample source module called LIBINIT3 such that the name of the load module representing the actual INMOD is specified as the first argument of the CEEXPITY macro, as follows: CEEXPITY CINMODCL,0 In the above example, the name of the load module representing the actual INMOD is CINMODCL. 2 Assemble the supplied sample source module called LIBINIT3. Use the high-level assembler (batch or interactive) with default attributes for this purpose. 3 Link-edit the object code derived from Step 2 into an executable load module. The name of this load module must be different from the name of the load module representing the actual INMOD. Use the linkage editor or binder (batch or interactive) with default attributes for this purpose. 4 Within the utility script, substitute the name of the load module representing the interface (LIBINIT3) where the name of the load module representing the actual INMOD would be specified. The flow of control is as follows: utility --> interface --> inmod For example: FASTLOAD --> LIBINIT3 --> CINMODCL 224 Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines UNIX OS UNIX OS SPARC Systems Running Oracle Solaris Use the following syntax to compile source files into a shared object module for INMOD or notify exit routines on a SPARC client system running Solaris: cc -G sourcefile -KPIC -o shared-object-name 2409A051 where Syntax Element Description cc Call to the program that invokes the native UNIX C compiler -G Linker option that generates a shared object file -KPIC Compiler option that generates Position Independent Code (PIC) for all user exit routines -o Switch to the linker shared-object-name Name of the shared object file This is the name specified as the: • INMOD= name parameter in the DEFINE command of the Teradata FastLoad job script • EXIT name parameter of the NOTIFY command of the Teradata FastLoad job script The shared-object-name can be any valid UNIX file name. Note: When creating a shared object module for an INMOD routine, if the INMOD uses functions from an external library, then that library must be statically linked with the INMOD routine so that the Teradata FastLoad utility can resolve the external references. sourcefile UNIX file name(s) of the source file(s) for the INMOD or notify exit routine Opteron Systems Running Oracle Solaris Use the following syntax to compile source files into a shared object module for INMOD or notify exit routines on an Opteron client system running Solaris: cc -dy -G sourcefile.c -o shared-object-name 2409A055 where Teradata FastLoad Reference 225 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines UNIX OS Syntax Element Description cc Call to the program that invokes the native UNIX C compiler -dy Specifies to use dynamic linking -G Specifies to create a shared object sourcefile Is a C source module for the INMOD -o Specifies the output file name shared-object-name Specifies the resulting shared object module This is the name specified as: • The INMOD modulename parameter of the DEFINE command of the Teradata FastLoad job script • The EXIT name parameter for the NOTIFY command of the Teradata FastLoad job script The shared-object-name can be any valid UNIX file name HP-UX PA RISC Use the following syntax to compile and link source files into a shared object module for INMOD notify exit routines on HP-UX PA RISC client systems: Compile Syntax cc -Aa -D_HPUX_SOURCE +z +u1 -c sourcefile.c % Link Syntax ld -b objectfile.o -o shared-object-name 2411A002 where 226 Syntax Element Description -Aa Compiler option which enables compiler to conform to the ANSI standard -b Linker option that generates a shared object file -c Compile-only option (does not link) cc Call to the program that invokes the native UNIX C compiler Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines UNIX OS Syntax Element Description -D_HPUX_SOURCE Symbol that enables the compiler to access macros and typedefs that are not defined by the ANSI Standard but are provided by the HPUX Operating System ld Call to the program that invokes the native UNIX linker objectfile File that the compiler generates and linker uses to generate shared-object-name -o Switch to the linker shared-object-name Name of the shared object file This is the name specified as the: • INMOD modulename parameter of the DEFINE command • EXIT name parameter for the NOTIFY command of the Teradata FastLoad job script The shared-object-name can be any valid UNIX file name. Note: When creating a shared object module for an INMOD routine, if the INMOD uses functions from an external library, then that library must be statically linked with the INMOD routine so that the Teradata FastLoad utility can resolve the external references.name. +z Compiler option that generates Position Independent Code (PIC) for all user exit routines sourcefile UNIX file name(s) of the source file(s) for the INMOD or notify exit routine +u1 Compiler option that allows pointers to access non-natively aligned data HP-UX Itanium Use the following syntax examples to compile and link a C INMOD on an HP-UX Itanium-based client. Compile Syntax cc +u1 -D_REENTRANT +DD64 -c inmod.c 2409A057 where Syntax Element Description cc Invokes the native UNIX C compiler +u1 Is a compiler option that allows pointers to access non-natively aligned data Teradata FastLoad Reference 227 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines UNIX OS Syntax Element Description -D_REENTRANT Ensures that all the Pthread definitions are visible at compile time +DD64 Generates 64-bit object code for PA2.0 architecture -c Compiles one or more source files but does not enter the linking phase inmod.c A C source module for the INMOD Link Syntax ld -n -b inmod.o -lc inmod.so -o 2409A056 where Syntax Element Description ld Invokes the UNIX linker editor -n Generates an executable with file type SHARE_MAGIC This option is ignored in 64-bit mode. -b Is a linker option specified to generate a shared object file inmod.o Is an object module derived from the compile step -lc Search a library libc.a, libc.so, or libc.sh -o Specifies the output filename; default is a.out inmod.so Specifies the resulting shared object module This is the user-specified name in the IMPORT command. Note: Object modules can be linked into shared objects or shared libraries (for example, .so or .sl extension respectively) on HP-UX Itanium. Linux Use the following syntax to compile and link source files into a shared object module for INMOD, OUTMOD, or notify exit routines on Linux 32-bit client systems. On Linux 64-bit client systems compile in 32-bit mode. Note: Be sure to compile the INMOD and notify exit routines in 32-bit mode so they are compatible with Teradata FastLoad even if it is for a 64-bit Linux machine. Compile Syntax gcc -I/usr/include -shared -fPIC -m32 228 sourcefile.c -o shared-object-name 2409B023 Teradata FastLoad Reference Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines UNIX OS where: Syntax Element Description gcc Call to the program that invokes the native C compiler -shared Flag that produces a shared object that can then be linked with other objects to form an executable -m32 Generates code for a 32-bit environment. -fPIC Compiler option that generates Position Independent Code (PIC) for all user exit routines -o Output file name inmod.c An INMOD source file name inmod.so An INMOD shared object name z/Linux z/Linux To compile and link source files into a shared object module for INMOD and notify exit routines on z/Linux client systems, use the following syntax: JFF,XVULQFOXGHPVKDUHGI3,&VRXUFHILOHFRVKDUHGREMHFWQDPH $ where Syntax Element Description gcc Call to the program that invokes the native C compiler -m31 Generates code for a 32-bit environment. -shared Flag that produces a shared object that can then be linked with other objects to form an executable -fPIC Compiler option that generates Position Independent Code for all user exit routines -o Output file name sourcefile UNIX file name(s) of the source file(s) for the INMOD or notify exit routine shared-object-name Name of the shared object file This is the name specified as: • The INMOD modulename parameter of the DEFINE command of the Teradata FastLoad job script. • The EXIT name parameter for the NOTIFY command of the Teradata FastLoad job script. Teradata FastLoad Reference 229 Appendix D: Compile, Link, and Execute INMOD and Notify Exit Routines Windows Windows The Teradata FastLoad INMOD routine is implemented as a dynamic load library (.dll) in C. A make file blkexit.mak is included to allow this DLL to be modified and rebuilt. Enter this command: nmake -f blkexit.mak When the DLL, blkexit.dll, is rebuilt, put the DLL in the same directory as fastload.exe. 230 Teradata FastLoad Reference APPENDIX E User-Defined-Types and User-Defined-Methods This appendix provides information on User-Defined-types (UDFs) and User-Defined-Methods (UDMs): • User-Defined-Types and User-Defined-Methods • User-Defined-Methods (UDMs) • Creating UDTs with FastLoad • Inserting and Retrieving UDTs with Client Products • External Types • Inserting UDTs with FastLoad • Retrieving UDTs with FastLoad • Retrieving UDT Metadata with FastLoad User-Defined -Types and User-Defined -Methods This section provides a brief overview of how Teradata Database client products support user-defined custom data types known as User-Defined Types (UDTs), and user-defined custom functions known as User-Defined Methods (UDMs). • User-Defined Types (UDTs) • User-Defined Methods (UDMs) For more in depth information on using UDTs and UDMs, see Introduction to Teradata (B035-1091), SQL External Routine Programming (B035-1147) and SQL Fundamentals (B035-1141). User-Defined-Types (UDTs) UDTs can be created to provide representations of real world entities within the Teradata Database. Choosing a set of UDTs which closely matches the entities encountered in a given business can yield a system which is easier to understand and hence easier to maintain. Support for UDTs and UDMs integrates the power of object-oriented technology directly into the Teradata Database. Teradata FastLoad Reference 231 Appendix E: User-Defined-Types and User-Defined-Methods User-Defined-Types and User-Defined-Methods User-Defined-Methods (UDMs) Teradata Database developer-created custom functions, which are explicitly connected to UDTs are known as User-Defined-Methods (UDMs). All UDMs must reside on server. UDMs can be used to create an interface to the UDT that is independent of the UDT's internal representation. This makes it possible to later enhance a given UDT even in cases where the internal representation of the UDT must be changed to support the enhancement, without changing all of the database applications which use the UDT. Creating UDTs with FastLoad Teradata FastLoad cannot create a custom UDT. Inserting and Retrieving UDTs with Client Products A UDT can only exist on the Teradata Database server. Each UDT has an associated “from-sql routine” and “to-sql routine”. • “insert – The “to-sql routine” constructs a UDT value from a pre-defined type value. The “to-sql routine” is automatically invoked when inserting values from a client system into a UDT on the Teradata Database server. • Retrieve – The “from-sql routine” generates a pre-defined type value from a UDT. The “from-sql routine” is automatically invoked when a UDT is retrieved from the Teradata Database server to a client system. External Types The “from-sql routine” and the “to-sql routine” create a mapping between a UDT and a pre-defined type. This pre-defined type is called the external type of a UDT. A client application only deals with the external type; it does not deal with UDT value directly. External Type Example For example, if the following conditions exist: • UDT named FULLNAME exists • The external type associated with FULLNAME is VARCHAR(46) Then, during an retrieve of FULLNAME values, the Teradata Database server converts the values from FULLNAME values to VARCHAR(46) values by invoking the “from-sql routine” associated with the FULLNAME UDT. Note: The client must expect to receive the data in the same format as it receives VARCHAR(46) values. Similarly, when values are provided by the client for insert into a FULLNAME UDT, the client must provide values in the same way it would provide values for a VARCHAR(46) field. The Teradata Database server will convert the values from VARCHAR(46) to FULLNAME values using the “to-sql routine” associated with the FULLNAME UDT. 232 Teradata FastLoad Reference Appendix E: User-Defined-Types and User-Defined-Methods User-Defined-Types and User-Defined-Methods Inserting UDTs with FastLoad Teradata FastLoad inserts values into tables containing UDT columns in the same manner as is done for other tables, and handles the column data in its external type format. • Data in External Type Format – Data must be in the external type format in the input data source identified in the DEFINE command in the Teradata FastLoad job script. • Name of the External Type – Specify the name of the external type for the data type of the field in the DEFINE command. Retrieving UDTs with FastLoad UDTs cannot be retrieved with FastLoad Retrieving UDT Metadata with FastLoad UDT Metadata cannot be retrieved with FastLoad. Teradata FastLoad Reference 233 Appendix E: User-Defined-Types and User-Defined-Methods User-Defined-Types and User-Defined-Methods 234 Teradata FastLoad Reference Glossary A ABORT: In Teradata SQL, a statement that stops a transaction in progress and backs out changes to the database only if the conditional expression associated with the abort statement is true. access lock: A lock that allows selection of data from a table that may be locked for write access. The Teradata MultiLoad utility maintains access locks against the target tables during the Acquisition Phase. access module: A software component that provides a standard set of I/O functions to access data on a specific device. account: The distinct account name portion of the system account strings, excluding the performance group designation. Accounts can be employed wherever a user object can be specified. acquisition phase: Responsible for populating the primary data subtables of the work tables. Data are received from the host, converted into internal format, and inserted into the work tables. The work tables will be sorted at the end of the Acquisition Phase and prior to the Application Phase. administrator: A special user responsible for allocating resources to a community of users. allocation group: (AG) A set of parameters that determine the amount of resources available to the sessions assigned to a PG referencing a specific AG. Has an assigned weight that is compared to other AG weights. An AG can limit the total amount of Central Processing Unit (CPU) used by sessions under its control. ANSI: American National Standards Institute. ANSI maintains a standard for SQL. For information about Teradata compliance with ANSI SQL, see SQL Fundamentals (B035-1141). Application Phase: Responsible for turning rows from a work table into updates, deletes, and inserts and applying them to a single target table. API: Application Program Interface. An interface (calling conventions) by which an application program accesses an operating system and other services. An API is defined at source code level and provides a level of abstraction between the application and the kernel (or other privileged utilities) to ensure the portability of the code. An API can also provide an interface between a high level language and lower level utilities and services written without consideration for the calling conventions supported by compiled languages. In this case, the API may translate the parameter lists from one format to another and the interpret call-by-value and call-by-reference arguments in one or both directions. Teradata FastLoad Reference 235 Glossary architecture: A definition and preliminary design which describes the components of a solution and their interactions. An architecture is the blueprint by which implementers construct a solution which meets the users’ needs. ASCII: American Standard Code for Information Interchange, a character set used primarily on personal computers. B BTEQ: Basic Teradata Query facility. A utility that allows users on a workstation to access data on a Teradata Database, and format reports for both print and screen output. C capture: The process of capturing a production data source. change data capture: The process of capturing changes made to a production data source. Change data capture is typically performed by reading the source DBMS log. It consolidates units of work, ensures data is synchronized with the original source, and reduces data volume in a data warehousing environment. mainframe-attached: A mainframe computer that communicates with a server (for example, a Teradata Database) through a channel driver. character set: A grouping of alphanumeric and special characters used by computer systems to support different user languages and applications. Various character sets have been codified by the American National Standards Institute (ANSI). Client: A computer that can access the Teradata Database. column: In the relational model of Teradata SQL, databases consist of one or more tables. In turn, each table consists of fields, organized into one or more columns by zero or more rows. All of the fields of a given column share the same attributes. COP: Communications Processor. One kind of interface processor (IFP) on the Teradata Database. A COP contains a gateway process for communicating with workstations via a network. COP Interface: Workstation-resident software and hardware, and Teradata Database-resident software and hardware, that allows workstations and the Teradata Database to communicate over networks. D database: A related set of tables that share a common space allocation and owner. A collection of objects that provide a logical grouping for information. The objects include, tables, views, macros, triggers, and stored procedures. Data Dictionary: In the Teradata Database, the information automatically maintained about all tables, views, macros, databases, and users known to the Teradata Database system, including information about ownership, space allocation, accounting, and privilege 236 Teradata FastLoad Reference Glossary relationships between those objects. Data Dictionary information is updated automatically during the processing of Teradata SQL data definition statements, and is used by the parser to obtain information needed to process all Teradata SQL statements. data manipulation: In Teradata SQL, the statements and facilities that change the information content of the database. These statements include INSERT, UPDATE, and DELETE. Data Model: A logical map that represents the inherent properties of the data independent of software, hardware, or machine performance considerations. The model shows data elements grouped into records, as well as the association around those records. data streams: Buffers in memory for temporarily holding data. A data stream is not a physical file; instead, it is more like a pipe (in UNIX or Windows systems), or a batch pipe in MVS. Data Warehouse: A subject oriented, integrated, time-variant, non-volatile collection of data in support of management’s decision making process. A repository of consistent historical data that can be easily accessed and manipulated for decision support. DBA: DD: Database Administrator Data dictionary or data definition. DEFINE Statement: A statement preceding the INSERT statement that describes the fields in a record before the record is inserted in the table. This statement is similar to the SQL USING clause. delimiter: In Teradata SQL, a punctuation mark or other special symbol that separates one clause in a Teradata SQL statement from another, or that separates one Teradata SQL statement from another. DLL: Dynamic-link library. A feature of the Windows family of operating systems that allows executable routines to be stored separately as files with .dll extensions and to be loaded only when needed by a program. DML: Data manipulation language. In Teradata SQL, the statements and facilities that manipulate or change the information content of the database. These statements include SELECT, INSERT, UPDATE, and DELETE. domain name: A group of computers whose host names (the unique name by which a computer is known on a network) share a common suffix, that is the domain name. DSN: Digital Switched Network. The completely digital version of the PSTN. Duplicate Row Check: A logic within the Teradata Database used to check for duplicate rows while processing each primary data row for INSERTs and UPDATEs. DWM: Dynamic Workload Manager. The product described in this document, which manages access to the Teradata Database. Teradata FastLoad Reference 237 Glossary E EBCDIC: Extended binary coded decimal interchange code. An IBM code that uses 8 bits to represent 256 possible characters. It is used primarily in IBM mainframes, whereas personal computers use ASCII. Error Tables: Tables created during the Preliminary Phase used to store errors detected while processing a MultiLoad job. There are two error tables, ET and UV, that contains errors found during the Acquisition Phase and Application Phase, respectively. EOF: End of File EON: Extended Object Names execution time frame: waiting to run. A period of time when DWM can execute scheduled requests that are Exit Routines: Specifies a predefined action to be performed whenever certain significant events occur during a MultiLoad job. F FastExport: Teradata FastExport utility. A program that quickly transfers large amounts of data from tables and views of the Teradata Database to a client-based application. FastLoad: Teradata FastLoad utility. A program that loads empty tables on the Teradata Database with data from a network-attached or mainframe-attached client. field: The basic unit of information stored in the Teradata Database. A field is either null, or has a single numeric or string value. See also column, database, row, table. foreign key: The primary key of a parent data subject that is placed in a subordinate data subject. Its value identifies the data occurrence in the parent data subject that is the parent of the data occurrence in the subordinate data subject. formatted records: See Records. function: User Defined Functions (UDFs) are extensions to Teradata SQL. Users can write UDFs to analyze and transform data already stored in their data warehouse in ways that are beyond the functionality of Teradata’s native functions. G gateway: A device that connects networks having different protocols. global rule: Object Access and Query Resource rules can be specified as being global, that is, they apply to all objects, and therefore to all requests. When a rule is specified as being global, no query objects need be (or can be) associated with the rule because all objects are implicitly included. Care should be taken defining a global access rule, as it causes all requests to be rejected except those from the DBC user and any bypassed objects. 238 Teradata FastLoad Reference Glossary globally distributed objects (GDO): A data structure that is shared by all of the virtual processors in the Teradata Database system configuration. I import: This refers to the process of pulling system information into a program. To add system information from an external source to another system. The system receiving the data must support the internal format or structure of the data. Import Task: A task that quickly applies large amounts of client data to one or more tables or views on the Teradata Database. Composed of four major phases: Preliminary, Acquisition, Application, and End. The phases are a collection of one or more transactions that are processed in a predefined order according to the MLOAD protocol. An import task references up to five target tables. INMOD: Input Module, a program that administrators can develop to select, validate, and preprocess input data. INMOD Routine: User-written routines that Teradata MultiLoad and other load/export utilities use to provide enhanced processing functions on input records before they are sent to the Teradata Database. Routines can be written in C language (for network-attached platforms), or COBOL, PL/I or Assembler (for mainframe-attached platforms). A routine can read and preprocess records from a file, generate data records, read data from other database systems, validate data records, and convert data record fields. inner join: In Teradata SQL, a join operation on two or more tables, according to a join condition, that returns the qualifying rows from each table. instance: In object-oriented programming, refers to the relationship between an object and its class. The object is an instance of the class. In Teradata PT, an instance is an occurrence of a fully defined Teradata PT operator, with its source and target data flows, number of sessions, and so on. Teradata PT can process multiple instances of operators. interface processor (IFP): Used to manage the dialog between the Teradata Database and the host. Its components consist of session control, client interface, the parser, the dispatcher, and the BYNET. One type of IFP is a communications processor (COP). A COP contains a gateway process for communicating with workstations via a network. internet protocol (IP): Data transmission standard; the standard that controls the routing and structure of data transmitted over the Internet. I/O: Input/output. ISO: International Standards Organization J JCL: Job Control Language is a language for describing jobs (units of work) to the OS/390, z/OS, and VSE operating systems, which run on IBM's OS/390 and z800/900 large server (mainframe) computers. These operating systems allocate their time and space resources Teradata FastLoad Reference 239 Glossary among the total number of jobs that have been started in the computer. Jobs in turn break down into job steps. All the statements required to run a particular program constitute a job step. Jobs are background (sometimes called batch) units of work that run without requiring user interaction (for example, print jobs). In addition, the operating system manages interactive (foreground) user requests that initiate units of work. In general, foreground work is given priority over background work. JIS: Japanese Industrial Standards specify the standards used for industrial activities in Japan. The standardization process is coordinated by Japanese Industrial Standards Committee and published through Japanese Standards Association. Job Script: A job script, or program, is a set of Teradata MultiLoad commands and Teradata SQL statements that make changes to specified target tables and views in the Teradata Database. These changes can include inserting new rows, updating the contents of existing rows, and deleting existing rows. join: A select operation that combines information from two or more tables to produce a result. L LAN: Local Area Network. LANs supported by Teradata products must conform to the IEEE 802.3 standard (Ethernet LAN). Least Used: Least used (-lu) in a command line parameter that tells Teradata QD to route queries to the least used database. Load operator: A Teradata PT consumer-type operator that emulates some of the functions of the Teradata FastLoad utility in the Teradata PT infrastructure. LOB: An acronym for large object. A large object is a database object that is large in size. LOBs can be up to 2 gigabytes. There are two types of LOBs, CLOBs and BLOBs. CLOBs are character-based objects, BLOBs are binary-based objects. Locks: Teradata FastLoad automatically locks any table being loaded and frees a lock only after an END LOADING statement is entered. Therefore, access to a table is available when Teradata FastLoad completes. log: A record of events. A file that records events. Many programs produce log files. Often a log file determines what is happening when problems occur. Log files have the extension “.log”. logical action: A named action that is defined on the Alert Policy Editor's Actions tab. Logical actions can be assigned to events in the alert policy. Logical Data Model: A data model that represents the normalized design of data needed to support an information system. Data are drawn from the common data model and normalized to support the design of a specific information system. Actual implementation of a conceptual module in a database. It may take multiple logical data models to implement one conceptual data model. 240 Teradata FastLoad Reference Glossary loner value: A value that has a frequency greater than the total number of table rows divided by the maximum interval times 2. M macro: a file that is created and stored on the Teradata Database, and is executed in response to a Teradata SQL EXECUTE statement merge join: In Teradata SQL, the type of join that occurs when the WHERE conditional of a SELECT statement causes the system first to sort the rows of two tables based on a join field (specified in the statement), then traverse the result while performing a merge/match process. Metadata: Data about data. For example, information about where the data is stored, who is responsible for maintaining the data, and how often the data is refreshed. methods: In object-oriented programming, methods are the programming routines by which objects are manipulated. NFS: Network file system. MIB: Management Information Base MOSI: Micro Operating System Interface. A library of routines that implement operating system dependent and protocol dependent operations on the workstation. MTDP: Micro Teradata Director Program. A library of routines that implement the session layer on the workstation. MTDP is the interface between CLI and the Teradata Database. MultiLoad: Teradata MultiLoad. A command-driven utility that performs fast, high-volume maintenance functions on multiple tables and views of the Teradata Database. Multiset Tables: Tables that allow duplicate rows. MVS (Multiple Virtual Storage): One of the primary operating systems for large IBM computers. N name: A word supplied by the user that refers to an object, such as a column, database, macro, table, user, or view. nested join: In Teradata SQL, this join occurs when a field is a unique primary index for one table of the join and an index (unique or non-unique primary or secondary index) for the second table in the join. Network: In the context of the Teradata Database, a LAN (see LAN). network attached: A computer that communicates over the LAN with a server (for example, a Teradata Database). notify exit: A user-defined exit routine that specifies a predefined action to be performed whenever certain significant events occur during a Teradata FastLoad job. Teradata FastLoad Reference 241 Glossary For example, by writing an exit in C (without using CLIv2) and using the NotifyExit attribute in an operator definition, a routine to detect whether a Teradata FastLoad job succeeds or fails, how many records were loaded, what the return code is for a failed job, and so on can be provided. null: The absence of a value for a field. Nullif Option: This option allows the user to null a column in a table under certain conditions; it is only used in conjunction with DEFINE statements. NUSI: Non-unique secondary index; an NUSI is efficient for range query access, while a unique secondary index (USI) is efficient for accessing a single value. O object: In object-oriented programming, a unique instance of a data structure defined according to the template provided by its class. Each object has its own values for the variables belonging to its class and can respond to the messages, or methods, defined by its class. object access rule: An Object Access filter allows defining the criteria for limiting access to issuing objects and/or query objects. Queries that reference objects associated with the rule (either individually or in combination) during the specified dates and times are rejected. Global rules are not applicable for this type. object definition: The details of the structure and instances of the objects used by a given query. Object definitions are used to create the tables, views, and macros, triggers, join indexes, and stored procedures in a database. operator routine: method. In object-oriented programming, refers to a function that implements a The terms operator routine and operator function may be used interchangeably. OS/390 Operating System 390 outer join: In Teradata SQL, an extension of an inner join operation. In addition to returning qualifying rows from tables joined according to a join condition (the inner join), an outer join returns non-matching rows from one or both of its tables. Multiple tables are joined two at a time. owner: In Teradata SQL, the user who has the ability to grant or revoke all privileges on a database to and from other users. By default, the creator of the database is the owner, but ownership can be transferred from one user to another by the GIVE statement. P parameter: A variable name in a macro for which an argument value is substituted when the macro is executed. parser: A program executing in a PE that translates Teradata SQL statements entered by a user into the steps that accomplish the user’s intensions. 242 Teradata FastLoad Reference Glossary parsing engine (PE): An instance (virtual processor) of the database management session control, parsing, and dispatching processes and their data context (caches). Paused MultiLoad Job: A job that was halted, before completing, during the Acquisition Phase of the Teradata MultiLoad operation. The paused condition can be intentional, or the result of a system failure or error condition. peak perm: Highest amount of permanent disk space, in bytes, used by a table. performance groups: A performance group is a collection of parameters used to control and prioritize resource allocation for a particular set of Teradata Database sessions within the Priority Scheduler. Every Teradata Database session is assigned to a performance group during the logon process. Performance groups are the primary consideration in partitioning the working capacity of the Teradata Database. To learn more about performance groups, see Utilities (B035-1102). performance periods: A threshold or limit value that determines when a session is under the control of that performance period. A performance period links PGs/Teradata Database sessions under its control to an AG that defines a scheduling strategy. A performance period allows AG assignments based on time-of-day or resource usage to be changed. Physical Data Model: A data model that represents the denormalized physical implementation of data that support an information system. The logical data model is denormalized to a physical data model according to specific criteria that do not compromise the logical data model but allow the database to operate efficiently in a specific operating environment. PL/I: Programming Language/1, a programming language supported for MultiLoad development. Primary server: A Teradata server in which client applications execute transactions through use of Teradata SQL or utilities such as Teradata MultiLoad and update the tables of one or more replication groups. The changes are captured by Teradata replication services components and given to an Replication intermediary server and processes connected to the Primary server. priority definition set: A collection of data that includes the resource partition, performance group, allocation group, performance period type, and other definitions that control how the Priority Scheduler manages and schedules session execution. product join: In Teradata SQL, the type of join that occurs when the WHERE conditional of a SELECT statement causes the Teradata Database system to compare all qualifying rows from one table to all qualifying rows from another table. Because each row of one table is compared to each row of another table, this join can be costly in terms of system performance. Note that product joins without an overall WHERE constraint are considered unconstrained (Cartesian). If the tables to be joined are small, the effect of an unconstrained join on performance may be negligible, but if they are large, there may be a severe negative effect on system performance. Teradata FastLoad Reference 243 Glossary profiles: A profile is a set of parameters assigned to a user, group of users, or an account that determines which scheduling capabilities are available and how the Teradata Query Scheduler scheduled requests server handles their scheduled requests. physical action: A basic action type, such as <Send a Page>, <Send an E-Mail>, etc. Physical actions must be encapsulated by logical actions in order to be used in the alert policy. PIC: Position independent code PL/I: Programming Language/1, a programming language supported for MultiLoad development. PP2: Preprocessor2 PPP: Point-to-Point Protocol Primary Key: A set of one or more data characteristics whose value uniquely identifies each data occurrence in a data subject. A primary key is also known as a unique identifier. privilege: A user’s right to perform the Teradata SQL statements granted to him against a table, database, user, macro, or view. procedure: Short name for Teradata stored procedure. Teradata provides Stored Procedural Language (SPL) to create stored procedures. A stored procedure contains SQL to access data from within Teradata and SPL to control the execution of the SQL. production system: A database used in a live environment. A system that is actively used for day to day business operations. This differs from a test or development system that is used to create new queries or test new features before using them on the production system. Protocol: network. The rules for the format, sequence and relative timing of messages exchanged on a Q query analysis: A feature that estimates the answer set size (number of rows) and processing time of a SELECT type query. query: A Teradata SQL statement, particularly a SELECT statement. Query Director: A Teradata client application used to balance sessions between systems according to user provided algorithms. query management: The primary function of DWM is to manage logons and queries. This feature examines logon and query requests before they are dispatched for execution within the Teradata Database, and may reject logons, and may reject or delay queries. It does this by comparing the objects referenced in the requests to the types of DBA-defined rules. R Records: When using the Teradata MultiLoad utility, both formatted and unformatted records are accepted for loading. A formatted record, in the Teradata Database world, consists 244 Teradata FastLoad Reference Glossary of a record created by a Teradata Database utility, such as Basic Teradata Query (BTEQ), where the record is packaged with begin- and end-record bytes specific to the Teradata Database. Unformatted records are any records not originating on a Teradata Database, such as Fortran files. These files contain records that must be defined before loading onto the Teradata Database. Replication Group: A set of tables for which either data changes are being captured on a primary server or applied on a subscriber server. Replication Intermediary Server: The server and computer software process written by a third party which interfaces to one or more Teradata servers and initiates a change data capture or change data apply operation with the Teradata replication services components. Replication Services Components: A set of software functions implemented in the Teradata server thatcapture change data on the tables of a replication group and the Teradata Access Module for Oracle GoldenGate that interact with the Oracle GoldenGate software running on the replication intermediary server. For detailed information on the replication services components, see Teradata Replication Services Using Oracle GoldenGate and the SQL Data Definition Language. request: In host software, a message sent from an application program to the Teradata Database. resource partition: A collection of prioritized PGs related by their users’ associations. Has an assigned weight that determines the proportion of resources available to that partition relative to the other partitions defined for that Teradata Database. Restart Log Table: One of four restart tables the Teradata MultiLoad utility creates that are required for restarting a paused Teradata MultiLoad job. result: The information returned to the user to satisfy a request made of the Teradata Database. results table/file: In the Schedule Request environment, a results table or file is a database table or a Windows file into which result data for a schedule request that is not self-contained are stored. results file storage: A symbolic name to a root directory where scheduled requests results are stored. A file storage location can be mapped to a Windows root directory where results are stored. row: Whether null or not, that represent one entry under each column in a table. The row is the smallest unit of information operated on by data manipulation statements. RowID join: In Teradata SQL, this join occurs when one of the join tables has a non-unique primary index constant, and another column of that table matches weakly with a non-unique secondary index column of the second table. RT: Response Time RTF: Rich Text File Teradata FastLoad Reference 245 Glossary rule: Rules are the name given to the method used by DWM to define what requests are prohibited from being immediately executed on the Teradata Database. That is, the rules enforced by DWM provide the Query Management capabilities. run file: A script that is not contained within the SYSIN file, but rather executed through use of the .RUN BTEQ command. S scheduled requests: The capability to store scripts of SQL requests and execute them at a scheduled later time. schema: Schemas are used to identify the structure of the data. Producers have an output schema, to define what the source data will look like in the data stream. Consumers have an input schema, to define what will be read from the data stream. If the input and output schemas are the same, only define the schema once. script: A file that contains a set of BTEQ commands and/or SQL statements. Security token: A binary string generated by a server when a replication group is created or altered that must be input to secure a change data capture or apply operation. separator: A character or group of characters that separates words and special symbols in Teradata SQL. Blanks and comments are the most common separators. server: A computer system running the Teradata Database. Typically, a Teradata Database server has multiple nodes, which may include both TPA and non-TPA nodes. All nodes of the server are connected via the Teradata BYNET or other similar interconnect. Session: A session begins when the user logs on to the Teradata Database and ends when the user logs off the Teradata Database. Also called a Teradata Database session. session: In client software, a logical connection between an application program on a host and the Teradata Database that permits the application program to send one request to and receive one response from the Teradata Database at a time. skew: This value is calculated based on a single Database collection interval. If the Session Collection rate is 60, then the skew is calculated for a 60-second period. The value is calculated using 'current' data values. For example, the Max CPU used during the past 60 seconds relative to the Average used over that same 60 seconds: skew = 100 * (1 – avg / max) Source Database: The database from which data will be extracted or copied into the Data Warehouse. SQL: Structured Query Language. An industry-standard language for creating, updating and, querying relational database management systems. SQL was developed by IBM in the 1970s for use in System R. It is the de facto standard as well as being an ISO and ANSI standard. It is often embedded in general purpose programming languages. Programming language used to communicate with the Teradata Database. 246 Teradata FastLoad Reference Glossary SSO: Single sign-on, an authentication option that allows users of the Teradata Database on Windows 2000 systems to access the Teradata Database based on their authorized network usernames and passwords. This feature simplifies the procedure requiring users to enter an additional username and password when logging on to Teradata Database via client applications. statement: A request for processing by the Teradata Database that consists of a keyword verb, optional phrases, operands and is processed as a single entity. statistics: These are the details of the processes used to collect, analyze, and transform the database objects used by a given query. stored procedure: Teradata Version 2 Release 4 and later supports stored procedures. A stored procedure is a combination of SQL statements and control and conditional handling statements that run using a single call statement. Subscriber server: A Teradata server in which changes captured from a primary server by an intermediary are applied to tables that duplicate those of the primary. Replication services components executing on the servers provide the capture and apply functions. supervisory user: In Data Dictionary, a user who has been delegated authority by the administrator to further allocate Teradata Database resources such as space and the ability to create, drop, and modify users within the overall user community. T table: A set of one or more columns with zero or more rows that consist of fields of related information. Target Database: The database in which data will be loaded or inserted. Target table: A user table where changes are to be made by a Teradata MultiLoad task. TCP/IP: Transmission Control Protocol/Internet Protocol. Teradata SQL: The Teradata Database dialect of the relational language SQL, having data definition and data manipulation statements. A data definition statement would be a CREATE TABLE statement and a data manipulation statement would be a data retrieval statement (a SELECT statement). TDP: Teradata Director Program; TDP provides a high-performance interface for messages communicated between the client and the Teradata system. TDPID: Teradata Director Program Identifier. The name of the Teradata Database being monitored. Target Level Emulation (TLE): Permits emulation of a target environment (target system) by capturing system-level information from that environment. The captured information is stored in the relational tables SystemFE.Opt_Cost_Table and SystemFE.Opt_RAS_Table. The information in these tables can be used on a test system with the appropriate column and Teradata FastLoad Reference 247 Glossary indexes to make the Optimizer generate query plans as if it were operating in the target system rather than the test system. test system: A Teradata Database where Optimizer-specific information is imported to emulate a target system and create new queries or test new features. title: In Teradata SQL, a string used as a column heading in a report. By default, it is the column name, but a title can also be explicitly declared by a TITLE phrase. TPA: Trusted Parallel Application. Transport: The process of extracting data from a source, interfacing with a destination environment, and then loading data to the destination. transaction: A set of Teradata SQL statements that is performed as a unit. Either all of the statements are executed normally or else any changes made during the transaction are backed out and the remainder of the statements in the transaction are not executed. The Teradata Database supports both ANSI and Teradata transaction semantics. trigger: One or more Teradata SQL statements associated with a table and executed when specified conditions are met. TTU: Teradata Tools and Utilities is a robust suite of tools and utilities that enables users and system administrators to enjoy optimal response time and system manageability with there Teradata system. Teradata FastLoad is included in Teradata Tools and Utilities. tuple: In a database table (relation), a set of related values one for each attribute (column). A tuple is stored as a row in a relational database management system. It is analogous to a record in a non relational file. type: An attribute of a column that specifies the representation of data values for fields in that column. Teradata SQL data types include numerics and strings. U UDF User Defined Functions UDM User-Defined Methods. The database developer can create custom functions that are explicitly connected to UDTs; these are known as UDMs. Functionalities directly applicable to a UDT can be located within the UDMs associated with that UDT rather than being replicated to all of the applications that use that UDT, resulting in increased maintainability. UDT A custom data type, known as a user-defined type. By creating UDTs, a database developer can augment the Teradata Database with data types having capabilities not offered by Teradata predefined (built-in) data types. Use Teradata FastLoad to import values into tables containing UDT columns in the same manner as is done for other tables. The input records to Teradata FastLoad must have the column data for UDT columns in its external type format. Unicode: A fixed-width (16 bits) encoding of virtually all characters present in all languages in the world. 248 Teradata FastLoad Reference Glossary unique secondary index (USI): One of two types of secondary indexes. A secondary index may be specified at table creation or at any time during the life of the table. It may consist of up to 16 columns. To get the benefit of the index, the query has to specify a value for all columns in the secondary index. A USI has two purposes: It can speed up access to a row which otherwise might require a full table scan without having to reply on the primary index, and it can be used to enforce uniqueness of a column or set of columns. user: In Teradata SQL, a database associated with a person who uses the Teradata Database. The database stores the person’s private information and accesses other Teradata Databases. UPI: Unique primary index; a UPI is required and is typically assigned to major entities in the database. user: A database associated with a person who uses the Teradata Database. The database stores the person’s private information and accesses other Teradata Databases. user groups: A group of users can be specified within DWM as either as a collection of individual users, or as all user names which satisfy a character string pattern (such as SALE*). The Teradata concept of roles is not used to define user groups, as it applies to privileges. User groups can generally be employed wherever an issuing object can be specified, and any condition applied to a group implicitly applies to all users within that group. UTF-8: In simple terms, UTF-8 is an 8 bit encoding of 16-bit Unicode to achieve an international character representation. In more technical terms, in UTF-8, characters are encoded using sequences of 1 to 6 octets. The only octet of a sequence of one has the higher-order bit set to 0, the remaining 7 bits are used to encode the character value. UTF-8 uses all bits of an octet, but has the quality of preserving the full US-ASCII range. The UTF-8 encoding of Unicode and UCS avoids the problems of fixed-length Unicode encodings because an ASCII file encoded in UTF is exactly same as the original ASCII file and all non-ASCII characters are guaranteed to have the most significant bit set (bit 0x80). This means that normal tools for text searching work as expected. UTF16 A 16-bit Unicode Translation Format. V Varbyte: A data type that represents a variable-length binary string. Varchar: A data type that represents a variable-length non-numeric character. Vargraphic: A data type that represents a variable-length string of characters. view: An alternate way of organizing and presenting information in a Teradata Database. A view, like a table, has rows and columns. However, the rows and columns of a view are not directly stored by the Teradata Database. They are derived from the rows and columns of tables (or other views) whenever the view is referenced. VM (Virtual Machine): VM/CMS Teradata FastLoad Reference One of the primary operating systems for large IBM computers. Virtual Machine/Conversational Monitor System 249 Glossary W workgroups: Workgroups represent collections of related scheduled request work for users, user groups, or accounts. Each workgroup is assigned a maximum number of requests that can be executing from that workgroup simultaneously thereby ensuring that requests for all workgroups get a fair share of their scheduled work done within the execution time frames. workload limits rule A Workload Limits rule allows limiting the number of logon sessions and all-AMP queries, as well as reject or delay queries when workload limits are encountered. Which users, accounts, performance groups, or users within performance groups that are associated with this type of rule can be defined. Workstation: A network-attached client. Work Table: A table created during the Preliminary Phase used to store intermediate data acquired from the host during an MLOAD task. These data will eventually be applied to a target table. Write Lock: A write lock enables a single user to modify a table. The Teradata MultiLoad utility maintains write locks against each target table during the Application Phase, and work tables and error tables for each task transaction. X XML: XML is the eXtensible Markup Language, a system created to define other markup languages. For this reason, it can also be referred to as a metalanguage. XML is commonly used on the Internet to create simple methods for the exchange of data among diverse clients. Z z/Linux: Linux operating system (RedHat or SuSE) compiled to run on IBM System z machines. z/OS (MVS (Multiple Virtual Storage) ) One of the primary operating systems for large IBM computers. 250 Teradata FastLoad Reference Index Symbols C .* default VALUES clause 125 * (asterisk character) 151 c specification 155, 156 character set ASCII 55 AXSMOD 57 determining 162 EBCDIC 55 kanji 55 programming considerations 55 supported 19 Unicode 57 UTF-16 57 UTF-8 57 CHARACTERS data type specification 105 character-to-date data conversions 20 character-to-numeric data conversions 20 CHARSET configuration file specification 53 CHECKPOINT keyword 92, 93 checkpoints description 20 internal 117 trade-offs 58 with access modules 58 CLEAR command defined 25 syntax 95 using with HELP TABLE commands 124 cname specification INSERT statement 125 COBOL program examples compiling, linking and executing on z/OS systems 219 INMOD routine 194 commands defined 24 functions 24 OS command specification 142 syntax, see individual commands comments 58 compiling and linking routines HP-UX 226 Linux 228 Opteron 225 Solaris 229 UNIX OS 225, 226, 227, 229 Numerics 3944 error message 59 A abort, defined 235 access privileges, required 126 accounts, defined 235 acctid specification 134 administrator, defined 235 ANSI/SQL DateTime data types 57 DEFINE command 108 INSERT statement 126 specifications 57, 108 ANSIDATE keyword 96 API, defined 235 ASCII character set code 55 specification 161 Assembler program examples compiling, linking and executing on z/OS systems 217 INMOD routine 191 AXSMOD character set 57 checkpoint support 58 command defined 25 syntax 89 B BEGIN LOADING command defined 25 example 94 syntax 91 BINARY specification 155 BLKEXIT.C INMOD routine, calling 71 BUFSIZE, configuration file specification 53 bypass-label (BPL) tapes, not supported 24 BYTE data type specification 104 BYTEINT data type specification 105 Teradata FastLoad Reference 251 Index z/Linux 229 concatenated data sets, not supported 24 concurrent load utility tasks, restrictions and limitations 59 configuration file 53 contents 53 file name and location 53 optional specification 33 overriding internal utility defaults 52 processing by the FastLoad utility 54 specifications 52 using 52 configuration parameters overridden by runtime parameters 53 BUFSIZE 36 CHARSET 37 ERRORLOG 38 MAXSESS 38 MINSESS 38 SLEEP 38 TENACITY 39 VERBOSE 39 overridden by utility commands 53 SESSIONS 151 SET SESSION CHARSET 161 SLEEP 167 TENACITY 170 CREATE TABLE statement defined 26 D data formats 157 handling, defined 24 length field, input record 21 transfer capabilities 19 type specifications 99 data conversions character-to-date 20 character-to-numeric 20 date-to-character 20 example 59 integer-to-decimal 20 limiting factors 59 numeric-to-numeric 20 overview 20 data dictionary, defined 236 data encryption runtime option specification 45 data manipulation, defined 237 data types, ANSI/SQL DateTime 57 DATABASE statement, defined 26 DATAENCRYPTION BEGIN LOADING command 92 252 configuration file setting 53 datatype specification 98 DATEFORM command defined 25 syntax 96 DateTime specifications 57 table 108 date-to-character data conversions 20 dbname specification BEGIN LOADING command 91 HELP TABLE command 123 DDNAME=filename data source specification 97, 99 decimal format 59 zoned 20 DECIMAL data type specification 105 DEFINE command defined 25 syntax 97 using with HELP TABLE command 123 using with INSERT statements 126 DELETE statement, defined 26 delimiters, defined 237 DISPLAY_ERRORS keyword specification 156 DROP TABLE statement, defined 26 duplicate rows not inserted by the FastLoad utility 18, 60 DWM defined 237 E EBCDIC character set codes 55 END LOADING command defined 25 syntax 117 end-loading phase 117 end-of-record field, input record 22 endrecordnumber specification 147 ERRLIMIT command defined 25 syntax 119 error tables correcting errors recorded in 82 dropped if empty 117 format of 80 multiple databases 91 reusing table names 93 ERRORFILES keyword 91 errorname1 specification 92, 93 errors correcting 80 in the first error table 80 in the second error table 82 Teradata FastLoad Reference Index data conversion 59 error recording 79 error table format 79, 93 handling 78 NOTIFY implications 140 insertion errors limiting 119 restarting after 119 specifying the ERRLIMIT value 60 errortname1 specification 92 errortname2 specification 93 execution time frame, defined 238 EXIT name specification 137, 138 exit routines, creating for NOTIFY 140 F FastLoad job scripts programming considerations 52 checkpoint trade-offs 58 data conversion factors 59 defined triggers limitation 63 nonunique index sorts 60 range constraints 60 record mode load anomaly 61 referential integrity 63 session limits 62 space requirements and limitations 62 writing 72 FastLoad utility character set specifications 55 data conversion capabilities 20 data transfer capabilities 19 file requirements 33 file types supported 20 formatted data 21 input data formats 20 input record fields 21 invoking 33 in batch mode 34 in interactive mode 34 operating features and capabilities 18 operating modes 19 overview 17 programming considerations character set specifications 55 comments 58 restarting 147 NOTIFY implications 140 starting 135 terminating 47 abort 48 normal 48 unformatted data 22 Teradata FastLoad Reference unsupported data sets and tapes 24 variable-length text 24 field defined 238 values, inserting 126 fieldname specification DEFINE command 97 INSERT statement 125 file types supported 20 FILE=filename data source specification 97, 99 fileid specifications RUN command 149 usage rules for z/OS 149 FLOADCFG, configuration file name 53 floadcfg.dat, configuration file name 53 foreign key references 62 formatted data 21 FORMATTED keyword specification 155 G global rule, defined 238 GRAPHIC data type specifications 106 DEFINE command 103 H HELP command defined 24 syntax 121 HELP TABLE command defined 24 syntax 123 HIGH specification 137 hours specification 170 I IBM C program examples compiling, linking and executing on z/OS systems 220 INMOD routine 197 indicator bit positions, illustration 94 bytes field, input record 21 INDICATORS keyword 92, 94 init-string specification 89 INMOD routines and programs Assembler program listing 191 BLKEXIT.C, calling 71 COBOL program listing 194 compiling, linking and executing Assembler on z/OS 217 C on Windows 230 COBOL on z/OS 219 253 Index IBM C on z/OS 220 PL/I on z/OS 222 creating your own 72 definition 64 entry points 65 examples 70 FastLoad-to-INMOD interface 66 IBM C program listing 197 INMOD type specification 44 INMOD-to-FastLoad interface 67 PL/I program listing 196 platforms and programming languages 64 programming structure 65 using 64 INMOD=name specification 99 INMODRETURN configuration file specification 53 inner join, defined 239 input data fields, input record 21 record fields 21 INSERT statement 125 defined 27 insertion errors limiting 119 restarting after 119 integer, CHECKPOINT specification 91, 92, 93 INTEGERDATE keyword 96 integer-to-decimal data conversions 20 INTO tname specification 125 invalid record numbers 148 J JIS, defined 240 job scripts example 73 multifile example first output file 181 second output file 184 third output file 186 programming considerations, error limits 60 JOB, SCOPE parameter specification 138 join index, restrictions 62 join, defined 240 K kanji character set codes 55 KANJIEUC_0U specification 161 KANJISJIS_0S specification 161 L Linux, compiling and linking routines 228 254 loading 63 LOGDATA command syntax 129 LOGMECH command syntax 130 LOGOFF command defined 24 syntax 131 LOGON command defined 25 syntax 133 LOW specification 137 M max, session specification 151 maximum sessions 44 MAXSESS 54 MEDIUM specification 137 merge join, defined 241 min, sessions specification 151 minimum sessions 44 MINSESS configuration file setting 53 description 54 minutes specification 167 MSG string specification 138 MSG string specification, NOTIFY command 138 multifile FastLoad jobs output file examples first output file 181 second output file 184 third output file 186 running 75 multiple databases 91 MULTISET tables duplicate rows not inserted 18, 60 using the MultiLoad utility 31 N name specification AXSMOD command 89 DEFINE command 97 NOTIFY command 138 name, defined 241 Named Pipes Access Module 89 nested join, defined 241 new-line characters in ASCII text files, accommodating 22 NFS. See Network File System No Primary Index tables 63 NOBLOCK parameter 138 nonlabel (NL) tapes, not supported 24 nonunique index sorts 60 secondary indexes 31 NoPI tables 63 Teradata FastLoad Reference Index NOSTOP keyword specification 156 NOTIFY command defined 25 syntax 137 null, defined 242 NULLIF option 98 numeric fields 107 numeric-to-numeric data conversions 20 O Object Access filter, defined 242 OFF specification 137 OLE DB Access Module 89 operating modes, supported 19 option specification 138 order of preference, utility variables 53 OS command, defined 25 outer join, defined 242 output buffer size specification 36 owner, defined 242 P parser, defined 242 parsing engine (PE), defined 243 password specification 133 paused FastLoad jobs definition 49 restarting 49 after a database overfill 50 after a system or FastLoad failure 49 after a Teradata Database failure 50 after an unrecoverable error 50 factors affecting 51 performance group, defined 243 PL/I program examples compiling, linking and executing on z/OS systems 222 INMOD routine 196 privileges, required 93 procedures, defined 244 product join, defined 243 product version numbers 3 profiles, defined 244 Q queries, defined 244 query analysis, defined 244 query management, defined 244 QUEUE option specification description 138 NOTIFY command 138, 139 QUIT command Teradata FastLoad Reference defined 24 syntax 145 R range constraints 60 examples 61 types 60 RECORD command defined 25 entering before INSERT statements 147 syntax 147 record mode load anomaly 61 record numbers, invalid 148 redundant conversions example 59 supported 20 referential integrity 63 request, defined 245 required privileges BEGIN LOADING command 93 INSERT statement 126 restart log table 93 restrictions and limitations, concurrent load tasks 59 results defined 245 files 245 tables 245 return codes 48 RNAME parameter 138 row, defined 245 RowID join, defined 245 rule, defined 246 RUN command defined 26 executing 150 nesting 150 syntax 149 S sample procedure, invoking FastLoad on z/OS systems 46 scheduled requests, defined 246 SCOPE parameter 138 secondary indexes not supported by the FastLoad utility 18, 19, 63 using the MultiLoad utility 31 semicolon characters FastLoad commands 87 SQL statements 87 separator, defined 246 session character set, AXSMOD 57 sessions 152 control commands, defined 24 255 Index defined 246 limits 62, 153 maximum 44 minimum 44 number of 135 invalid 153 reported 136, 153 optimal number 152 SESSIONS command defined 25 entering before the LOGON command 152 syntax 151 SET RECORD command defined 26 syntax 154 SET SESSION CHARSET command defined 26 syntax 161 SHOW command defined 25 syntax 163 SHOW TABLE statement 126 SHOW VERSIONS command defined 25 syntax 165 single sign-on LOGON syntax 133 SLEEP configuration file specification 53 defined 25 runtime option specification 38, 44 syntax 167 software releases, supported 3 Solaris, compiling and linking routines 225, 229 space requirements and limitations 62 SQL, defined 246 standard error file 33 input file 33 name specification 43 output file 33 name specification 44 startrecordnumber specification 147 statement, defined 247 status codes FastLoad-to-INMOD interface 66 INMOD-to-FastLoad interface 67 stored procedures, defined 247 string specification 138 supervisory user, defined 247 syntax and use 125 Syntax, how to read 175 SYSTEM/SYSTEMS specification 138 256 T table definitions, using to define data 107 table, defined 247 TASM 25 tdpid specification 134 TENACITY command defined 25 syntax 170 configuration file specification 53 runtime option specification 39, 45 Teradata SQL statements CREATE TABLE 26 DATABASE 26 DELETE 26 DROP TABLE 26 INSERT 27, 125 using BTEQ to enter 117 terminating FastLoad 47 termination return codes 48 TEXT string specification 138 TEXT keyword specification 155 THRU endrecordnumber specification 148 keyword, RECORD command 147 title, defined 248 tname specification BEGIN LOADING command 91 HELP TABLE command 123 INSERT statement 125 transaction, defined 248 triggers, defined limitation 63 troubleshooting 173 type, defined 248 U unformatted data 22 records 158 UNFORMATTED keyword specification 155 Unicode character sets 41, 42, 57 data, using 127 UNIX OS HP-UX Itanium 227 HP-UX PA RISC 226 Linux 228 SPARC Systems Running Oracle Solaris 225 z/Linux 229 unsupported data sets and tapes 24 user 249 user groups, defined 249 Teradata FastLoad Reference Index username specification 133 UTF-16 character sets 41, 42, 57 defined 249 UTF-8 character sets 41, 42 defined 249 utility variables, order of preference 53 V value, NULLIF specification 97 VALUES keyword 125 VARGRAPHIC data type specification 106 variable-length fields, using 159 text 24 VARTEXT keyword specification 155 records 158 version numbers 3 VERSIONS keyword 165 W WebSphere MQ Access Module for Teradata client version 89 server version 89 workgroups, defined 250 workload limits rule, defined 250 Z z/Linux, compiling and linking routines 229 z/OS program examples for creating and using INMOD routines in Assembler 217 in COBOL 219 in IBM C 220 in PL/I 222 for invoking FastLoad 46 INMOD routines using Assembler 191 using COBOL 194 using IBM C 197 using PL/I 196 zoned decimal 59 formats, supported 20 Teradata FastLoad Reference 257 Index 258 Teradata FastLoad Reference
