Download Aster Client Guide - Information Products

Aster Client Guide
Release Number AC 5.10
April 2013
The product or products described in this book are licensed products of Teradata Corporation or its affiliates.
Teradata, Active Data Warehousing, Active Enterprise Intelligence, Applications-Within, Aprimo, Aprimo Marketing Studio, Aster, BYNET,
Claraview, DecisionCast, Gridscale, MyCommerce, Raising Intelligence, Smarter. Faster. Wins., SQL-MapReduce, Teradata Decision Experts,
"Teradata Labs" logo, "Teradata Raising Intelligence" logo, Teradata ServiceConnect, Teradata Source Experts, "Teradata The Best Decision Possible"
logo, The Best Decision Possible, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the
United States and other countries.
Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.
AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.
Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation
in the United States and/or other countries.
Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access,
Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum
Support are servicemarks of Axeda Corporation.
Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.
GoldenGate is a trademark of Oracle.
Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.
Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other
countries.
Intel, Pentium, and XEON are registered trademarks of Intel Corporation.
IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation.
Linux is a registered trademark of Linus Torvalds.
LSI is a registered trademark of LSI Corporation.
Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States
and other countries.
NetVault is a trademark or registered trademark of Quest Software, Inc. in the United States and/or other countries.
Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.
Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates.
QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.
Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license.
SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.
SPARC is a registered trademark of SPARC International, Inc.
Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and
other countries.
Unicode is a registered trademark of Unicode, Inc. in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other product and company names mentioned herein may be the trademarks of their respective owners.
THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN "AS-IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION
MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL,
OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
The information contained in this document may contain references or cross-references to features, functions, products, or services that are not
announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions,
products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or
services available in your country.
Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated
without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time
without notice.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document.
Please email: [email protected].
Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata
Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform,
create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata
Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including
developing, manufacturing, or marketing products or services incorporating Feedback.
Copyright © 2000-2013 by Teradata Corporation. All Rights Reserved.
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Conventions Used in This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Typefaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
SQL Text Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Command Shell Text Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Contacting Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
About Teradata Aster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
About This Document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 1: Install Aster Database Tools and Utilities . . . . . . . . 11
Install the Aster Database Cluster Terminal (ACT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Obtain ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Install ACT on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Install ACT on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Install ACT on MacOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Install ACT on Solaris. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Chapter 2: Aster Database Cluster Terminal (ACT) . . . . . . . . . . . 15
ACT Quick Start. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Install ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Launch ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Launch ACT on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Launch ACT on Linux or Solaris. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Launch ACT on Mac. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Launch ACT Directly on the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Log In to ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Startup Parameters for ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Using the “on-error-stop” Option in ACT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Use a Configuration File to Pass ACT Startup Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 22
How to Use ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Issue SQL Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Aster Client Guide
3
Exit ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Page Through Query Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Throttle Query Results in ACT and Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
ACT Utility Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Repeat Previously Typed Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Tab Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
ACT Commands (at the SQL Prompt) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Database Parameters Set with \set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Common SSL Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Setting Parameters on the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Encrypting Communications from the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Using Single Sign-on (SSO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Scenario 1: Queen Provides a Self-Signed Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate . . . . . . . . . . 38
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate . . . . . . . . . . 39
Scenario 4: Encrypting Communication from the Queen to the Client . . . . . . . . . . . . . . 40
Troubleshooting ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
ACT Connection Hangs When Using SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Misleading Error Message Reports Problem With a Role Instead of With a User . . . . . . 41
Chapter 3: Connect Using Database Drivers . . . . . . . . . . . . . . . . . . . . 43
General Tips for Connecting Clients to Aster Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Recommended Character Set Is UTF-8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'. . . . . . . . . . . 44
AIX Client Dependent Libaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
ODBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
ODBC Driver Manager Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Obtain the ODBC Driver Package. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Install ODBC on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Install ODBC on Linux, Solaris, or MacOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Configure Driver Manager on Linux and AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Install and Configure the unixODBC Driver Manager on Solaris . . . . . . . . . . . . . . . . . . . 52
Set Up ODBC for Perl Connectivity on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Set up ODBC for PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
ODBC Usage Notes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Aster JDBC Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Differences from the Legacy JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Before You Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Install the JDBC Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4
Aster Client Guide
Use the JDBC Driver in a Java Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Parameters for Connecting through JDBC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Configuring the JDBC Log Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Behavior and Performance Settings for JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
The Copy, Install File, Uninstall File, and Download File Commands . . . . . . . . . . . . . . . 65
Using Client-Side Cursors in JDBC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Test JDBC Connect Program. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Configure Aster Database SQL Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
SQL Behavior Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Setting the SQL Behavior Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Syntax for ODBC Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Teradata Wallet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Wallet Contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
TD Wallet Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Download TD Wallet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Install and Configure TD Wallet on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Install and Configure TD Wallet on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
SSL Security for Aster Database-Client Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
SSL-Related Files and Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
SSL Settings Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Common SSL Configuration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Encrypting Data Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Adding AD-Based SSO Authentication to SSL-Secured Aster Database . . . . . . . . . . . . . . 84
How to Set Configuration Parameters on the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
How to Set Configuration Parameters on the Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Creating Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Process SQL Statements in JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Process a Simple Query in JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
JDBC Troubleshooting and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Connect Reporting Tools to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Connect Aqua Data Studio to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Connect MicroStrategy to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Chapter 4: Tools for .NET Environments . . . . . . . . . . . . . . . . . . . . . . . . . 97
Installing the Aster Database ADO.NET Driver (nClusterDNProvider) . . . . . . . . . . . . . . . . . 97
Installation Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Installing nClusterDNProvider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Sample Program to Demonstrate the Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Performing SSIS Data Loading with nClusterDNProvider . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Aster Client Guide
5
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Sample Program for ADO.NET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Features Demonstrated in the Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Steps to Build and Test the Sample Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Source Code of Program to Demonstrate nClusterDNProvider . . . . . . . . . . . . . . . . . . . 113
Possible Exceptions and Resolutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Chapter 5: Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Best Practices for Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Loading Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Scenario 1: Pre-Production Data Loading with Aster Database . . . . . . . . . . . . . . . . . . . . 122
Scenario 2: Loading in a Production Environment with Aster Database . . . . . . . . . . . . 123
Loading Best Practices Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Aster Database Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Argument Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Exit Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Install the Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Load Data with the Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Load from Multiple Files Using a Map File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Hints for Successful Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Error Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Troubleshoot Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Running Multiple Loaders Concurrently . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Load stalls upon cancellation or failure encountered during a load . . . . . . . . . . . . . . . . 143
Load fails on UNIQUE or PRIMARY KEY violation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Invalid Input Syntax Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Single quote character must be escaped when using the -q option . . . . . . . . . . . . . . . . . 143
Using the -C option with uppercase or special characters . . . . . . . . . . . . . . . . . . . . . . . . 144
Uppercase characters are passed as lowercase if not escape quoted . . . . . . . . . . . . . . . . . 144
Chapter 6: Export and Load Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Aster Database Export Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Argument Flags for Exporter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Installing Aster Database Export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6
Aster Client Guide
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Aster Client Guide
7
8
Aster Client Guide
Preface
This guide provides instructions for users and administrators of Aster Client, version AC 5.10.
If you’re using a later version, you must download a newer edition of this guide!
The following additional resources are available:
•
Aster Database upgrades, clients and other packages:
http://downloads.teradata.com/download/tools
•
Documentation for existing customers with a Teradata @ Your Service login:
http://tays.teradata.com/
•
Documentation that is available to the public:
http://www.info.teradata.com/
Conventions Used in This Guide
This document assumes that the reader is comfortable working in Windows and Linux/UNIX
environments. Many sections assume you are familiar with SQL.
This document uses the following typographical conventions.
Typefaces
Command line input and output, commands, program code, filenames, directory names, and
system variables are shown in a monospaced font. Words in italics indicate an example or
placeholder value that you must replace with a real value. Bold type is intended to draw your
attention to important or changed items. Menu navigation and user interface elements are
shown using the User Interface Command font.
SQL Text Conventions
In the SQL synopsis sections, we follow these conventions:
Aster Client Guide
•
Square brackets ([ and ]) indicate one or more optional items.
•
Curly braces ({ and }) indicate that you must choose an item from the list inside the braces.
Choices are separated by vertical lines (|).
•
An ellipsis (...) means the preceding element can be repeated.
9
Contacting Technical Support
•
A comma and an ellipsis (, ...) means the preceding element can be repeated in a commaseparated list.
•
In command line instructions, SQL commands and shell commands are typically written
with no preceding prompt, but where needed the default Aster Database SQL prompt is
shown: beehive=>
Command Shell Text Conventions
For shell commands, the prompt is usually shown. The $ sign introduces a command that’s
being run by a non-root user:
$ ls
The # sign introduces a command that’s being run as root:
# ls
Contacting Technical Support
For further assistance, contact Teradata technical support.
Support Portal: http://tays.teradata.com/
Email: [email protected]
Telephone: +1-650-273-5599
About Teradata Aster
Teradata Aster provides data management and advanced analytics for diverse and big data,
enabling the powerful combination of cost-effective storage and ultra-fast analysis of
relational and non-relational data. Teradata Aster is a division of Teradata and is
headquartered in San Carlos, California.
For more information, go to http://www.asterdata.com
About This Document
This is the “Aster Client Guide,” version AC 5.10, edition 1. This edition covers Aster Database
Clients version AC 5.10 and was published April 17, 2013 10:35 pm.
Document revision history:
December 2012: 5.0.3
April 2013: AC 5.10
Aster Client Guide
10
Install Aster Database Tools and
Utilities
CHAPTER 1
This section explains how to install various utilities that complement your Aster Database
installation.
•
Install the Aster Database Cluster Terminal (ACT) (page 11)
•
Install the Loader Tool (page 133)
Install the Aster Database Cluster Terminal
(ACT)
ACT is a terminal-based query tool that connects with Aster Database. ACT lets you type
queries, see the results, and save them to a file. Alternatively, you can source your queries from
a file. ACT supports connection via SSL and SSO and is available for Windows, Linux, Solaris
and Mac. Follow the instructions for your OS:
•
“Obtain ACT” on page 11
•
“Install ACT on Windows” on page 12
•
“Install ACT on Linux” on page 12
•
“Install ACT on MacOS” on page 13
•
“Install ACT on Solaris” on page 13
Obtain ACT
ACT is installed automatically on the queen during the installation of Aster Database. By
default, the install directory is /home/beehive/clients/act. To launch ACT from the
queen, see “Launch ACT Directly on the Queen” on page 17.
If you want to install ACT on another machine, get the file for the client operating system and
copy it to the computer you'll use to query your database. You may obtain the file in one of
two ways:
•
Aster Client Guide
To get the newest package, download it from http://downloads.teradata.com/download/tools
11
Install Aster Database Tools and Utilities
Install the Aster Database Cluster Terminal (ACT)
•
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
Install ACT on Windows
To install ACT on Windows:
1
Obtain ACT.
2
Uncompress the tar file and place the executable act.exe in a directory on your client
machine. Teradata Aster recommends that you create a directory, C:\Program
Files\asterdata and place the act.exe executable there.
3
Set permissions to make the file executable:
•
In Windows File Explorer, navigate to find the ACT executable file.
•
Right-click on it, select Properties, click Security.
•
In the To change permissions section, click Edit.
•
Click the name of the group who can use the application, and tick the checkbox for
Allow/Read & Execute in the bottom of the window.
•
4
Click OK and click OK again.
Test your ACT installation like this:
•
From the windows Start menu, choose Run..., type cmd, and click OK.
•
At the command line, change directories to the folder holding act.exe.
•
Type act --help at the command line. The help page should be printed to the
command line.
•
To connect to your Aster Database, type
act -h <ip address of your queen>
•
When prompted, type the database username and password (The default user/
password credentials are beehive/beehive) The <dbuser>=> prompt indicates you've
logged in successfully (i.e. beehive=> for the default user). Type \q to quit.
•
See the section “Launch ACT” on page 16 for information on running ACT.
Install ACT on Linux
To install ACT on Linux:
1
Obtain ACT.
2
Uncompress the tar file and place the executable in a directory on your client machine.
Teradata Aster recommends that you use the directory, /home/beehive/clients.
3
Change the file’s permissions to make it executable:
$ chmod 755 act
Tip! ACT for Linux requires glibc version 2.6.18 or higher. If you do not have glibc version 2.6.18 or higher, you must
use the IP address instead of the hostname for the -h flag when running ACT. To check the version of glibc, issue
the command ldd --version.
12
Aster Client Guide
Install Aster Database Tools and Utilities
Install the Aster Database Cluster Terminal (ACT)
4
See the section “Launch ACT” on page 16 for information on running ACT.
Install ACT on MacOS
To install ACT on MacOS:
1
Obtain ACT.
2
Double-click the .dmg file. A new icon with a name similar to the .dmg file name appears
on your desktop.
3
If a new Finder window doesn’t automatically appear, double click on the new icon that
has appeared on your desktop. A new Finder window appears.
4
Find the application’s icon in the Finder window. Drag and drop it into your Applications
directory.
5
See the section “Launch ACT” on page 16 for information on running ACT.
Install ACT on Solaris
To install ACT on Solaris:
1
Obtain ACT.
2
Use pkgadd to install ACT. Here is an example, using a SunOS 5.10 operating system:
$ pkgadd -d nCluster_AC 5.10_client_SunOS_5.10.pkg all
3
Aster Client Guide
See the section “Launch ACT” on page 16 for information on running ACT.
13
Install Aster Database Tools and Utilities
Install the Aster Database Cluster Terminal (ACT)
14
Aster Client Guide
Aster Database Cluster Terminal
(ACT)
CHAPTER 2
This section explains how to use Aster Database Cluster Terminal (ACT) to query and manage
databases. ACT is a terminal-based query tool that connects with Aster Database. ACT lets you
connect to the database (optionally using SSO and/or SSL), type queries, issue them to Aster
Database, and get query results. Alternatively, you can source your queries from a file. ACT
can return your query results to the command line or to a file, which makes it useful for
extracting data. Meta-commands and shell-like features are provided to facilitate writing
scripts and automating tasks.
Tip! Beginning with ACT version 4.6, the ACT client cannot connect to versions of Aster Database prior to version
4.6. If you attempt to connect to a pre-4.6 version of Aster Database with a 4.6 or later version of ACT, you will see an
error message indicating that there is a version mismatch between Aster Database and the client. You should obtain
the version of ACT that matches the version of Aster Database to which you are attempting to connect.
The following sections explain how to launch and use ACT:
•
ACT Quick Start (page 15)
•
Install ACT (page 16)
•
Launch ACT (page 16)
•
Startup Parameters for ACT (page 18)
•
How to Use ACT (page 24)
•
ACT Commands (at the SQL Prompt) (page 30)
•
Troubleshooting ACT (page 40)
ACT Quick Start
You run ACT from the command line. The launch command is:
$ act -d <db name> -h <hostname> -U <username>
[-w <password>] [argument flags]
where
Aster Client Guide
15
Aster Database Cluster Terminal (ACT)
Install ACT
•
db name is your database name,
•
hostname is the name or IP address of the queen,
•
username is your SQL login name,
•
password is your SQL password in Aster Database (this is optional; ACT prompts you for a
password if you do not pass a -w parameter), and
•
argument flags are any of the parameter/value combinations listed in “Startup Parameters
for ACT” on page 18.
For example:
$ act -d emea_sales -h 10.48.58.100 -U mjones
Tip! When using SSO (single sign-on), the -U and-w options are not used, because the username and password
are passed directly to the host via SSO.
To log in to the default database that is provided in your installation, type this, replacing the IP
address with the hostname or IP address of your Aster Database queen:
$ act -d beehive -h 10.42.52.100 -U beehive -w beehive
To see a list of ACT command line arguments, type:
$ act --help
Install ACT
You can install ACT on Linux, Windows, Solaris, or Mac.
Launch ACT
See the appropriate section below for instructions on launching ACT:
•
“Launch ACT on Windows” on page 17
•
“Launch ACT on Linux or Solaris” on page 17
•
“Launch ACT on Mac” on page 17
•
“Launch ACT Directly on the Queen” on page 17
Tip! On an Aster Database where LDAP authentication is enabled, if during logon an ACT user gets the error message:
'ERROR: An internal error has occurred.',
make sure the username is present in Aster Database with proper privileges.
•
16
“Log In to ACT” on page 18
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Launch ACT
Launch ACT on Windows
1
Make sure ACT is installed on your client machine.
2
Open a command prompt.
3
Change directories to the folder which contains the act.exe executable.
4
Log In to ACT.
Launch ACT on Linux or Solaris
1
Make sure ACT is installed on your client machine.
2
Ensure that your path includes the directory where ACT is installed.
3
Log In to ACT.
Tip! ACT for Linux requires glibc version 2.6.18 or higher. If you do not have glibc version 2.6.18 or higher, you must
use the IP address instead of the hostname for the -h flag when running ACT. To check the version of glibc, issue
the command ldd --version.
Launch ACT on Mac
1
Make sure ACT is installed on your client machine and that its directory is in your path.
2
Open a terminal.
3
Log In to ACT.
Launch ACT Directly on the Queen
Your Aster Database queen also contains an installation of the ACT client. To run it there,
you’ll need a user account on the queen machine, and you’ll need an SSH client on your
workstation.
To run the ACT on the queen:
1
Open a SSH connection to the queen. If you do not have a user account on the queen, ask
the machine’s administrator for one.
Tip! If you need an SSH client, do one of the following:
• Install the OpenSSH client as explained on the OpenSSH homepage at http://www.openssh.org/
• or, for Windows only, install the PuTTY SSH client.
2
Change directories to the directory where ACT is installed (by default, /home/beehive/
clients.
3
Log in to ACT:
$ act -d <db name> -U <username> -w <password> [argument flags]
Note that if you do not provide the hostname using -h, ACT defaults to the localhost. For
details on the command line options, see “Startup Parameters for ACT” on page 18
Aster Client Guide
17
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Log In to ACT
1
Run ACT by typing a command like:
act -d <db name> -h <hostname> -U <username> [-w <password>]
[argument flags]
For details on the command line options, see “Startup Parameters for ACT” on page 18
2
Provide your database password by:
•
adding –w <password> to the ACT login string, or
•
omitting the -w argument and providing your database password at the prompt.
3
Choose a database by adding -d <database name> to the ACT login string. If -d is not
used, ACT places you in the system database (with the default name “beehive”).
4
You will see a welcome message, followed by the database prompt, which shows the
database name, followed by “=>”. For example:
Welcome to act AC 5.10, the Aster Database Terminal.
beehive=>
Startup Parameters for ACT
ACT takes a variety of startup parameters when you launch it. (Don’t confuse these with
ACT’s at-the-SQL-prompt flags, which you can read about in “ACT Commands (at the SQL
Prompt)” on page 30.)
You can pass these parameters at the command-line (see “Using the “on-error-stop” Option in
ACT” on page 22) or in a configuration file or “Use a Configuration File to Pass ACT Startup
Parameters” on page 22). The same startup parameter cannot be repeated in a single
invocation of ACT, or an error will be returned.
To list the command line parameters, type act --help.
18
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Descriptions of each parameter are below:
Table 2 - 1: Summary of the most common command-line parameters for ACT
Flag
Description
-d [ --dbname ] DBNAME
Specify database name to connect to
(default: “beehive”).
-h [ --host ] HOSTNAME
Aster Database server host (default:
“localhost”).
-U [ --username ] NAME
Aster Database username (default:
“beehive”). Not used with SSO.
-l (the letter “ell”)
List available databases, then exit.
[ --list-databases ]
Tip! Note the default values for the connection parameters. If you do not specify the parameters -d (database
name), -h (hostname), -U (username), and/or -p (port) in the connect string, ACT will use the default values.
The default values are:
• “beehive” for database name
• “localhost” for hostname
• “beehive” for username, and
• “2406” for port.
If -w is not used, ACT will prompt for a password.
Table 2 - 2: General-purpose command-line parameters for ACT
Flag
Description
-d [ --dbname ] DBNAME
Specify database name to connect to (default:
“beehive”).
--config-file FILENAME
Lads startup parameters from a configuration file
specified by FILENAME. See “Use a Configuration File
to Pass ACT Startup Parameters” on page 22 for more
information.
-c [ --single-command
]COMMAND
Run only single command (SQL or internal) and exit.
For example:
act -c "COPY MyTable FROM stdin;" <
myDataFile.dat
-f [ --input-file ] FILENAME
Execute commands from file, then exit. Run a SQL
script.
-1 (the numeral “one”)
Execute command file as a single transaction.
[ --single-transaction ]
-l (the letter “ell”)
List available databases, then exit.
[ --list-databases ]
Aster Client Guide
-? [ --help ]
Show command line help, then exit.
-V [ --version ]
Output version information, then exit.
19
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 2 - 2: General-purpose command-line parameters for ACT (continued)
Flag
Description
--on-error-stop
Enables the “on-error-stop” option, by default this
option is disabled. See “Using the “on-error-stop”
Option in ACT” on page 22 for more information.
or
-E
Table 2 - 3: Input- and output-related command-line parameters for ACT
Flag
Description
-a [ --echo-script-input ]
Echo all input from script.
-e [ --echo-all-input ]
Echo commands sent to server.
-o [ --redirect-query-results ]
FILENAME
Send query results to file (or | pipe).
Table 2 - 4: SSL and SSO related command-line parameters for ACT
20
Flag
Description
--enable-ssl
Enables Secure Socket Layer (ssl) support. Must be used if
any of the other SSL/SSO arguments are used.
--ssl-encrypt-reads
SSL Encrypt Reads. Must be used if secureWrites=true
on the server. Conversely, must not be used if
secureWrites=false on the server. See How to Set
Configuration Parameters on the Queen (page 85) for
information on how to set the secureWrites parameter
on the server.
--ssl-self-signed-peer
Indicates that ACT will connect to a Queen which will
provide a self-signed certificate.
--ssl-private-key-path
PATH
The SSL Private Key Path indicates where the private key is
stored on the client (ACT) machine.
--ssl-certificate-path
PATH
The SSL Certificate Path indicates where the certificate is
stored on the client (ACT) machine.
--ssl-trusted-ca-dir
DIRECTORY
When using a chain of certificates rather than a single
certificate, use the SSL Trusted CA Dir to set the directory on
the client machine where the chain of trusted certificates is
stored.
--ssl-trusted-ca-file
FILENAME
Use SSL Trusted CA Filename to provide the location of the
signed copy of the server’s certificate on the client machine.
--ssl-cert-filetype ARG
SSL Certificate File Type (use 1 for PEM; 2 for ANS1; default:
0).
--enable-sso
Enables Single sign-on (SSO) support.
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 2 - 4: SSL and SSO related command-line parameters for ACT (continued)
Flag
Description
--gss-lib-path PATH
For Linux, sets the GSS shared library path (default on linux
is /opt/guest/lib32 or /opt/guest/lib64). Ignored on
Windows.
Tip! The SSL settings in ACT have interdependencies, and in most cases they rely on the SSL settings on the queen.
See Common SSL Configurations (page 36).
Table 2 - 5: Output format-related command-line parameters for ACT
Flag
Description
-q [ --quiet ]
Run quietly and do not print messages, only query
output. Use this for clean query output. Often used
with the -c flag.
-t [ --print-rows-only ]
Print rows only.
-x [ --expanded ]
Turn on expanded table output.
-A [ --unaligned ]
Turn on unaligned table output
-F [ --field-separator ] ARG
Set field separator (default: '|')
Table 2 - 6: Connection-related command-line parameters for ACT
Flag
Description
-h [ --host ] HOSTNAME
Aster Database queen hostname or IP address
(default: "localhost"). Note that ACT supports glibc
version 2.6.18 or higher. If you do not have glibc
version 2.6.18 or higher, you must use the IP address
instead of the hostname. To check the version of
glibc, issue the command ldd --version.
When using SSO, you should specify a fully qualified
hostname using the -h option, as in the example:
<hostname>.<domain>.<com|org etc>. If
only the hostname is used with SSO, ACT will
append the local domain name before attempting to
look up the host. Using an IP address with -h is not
supported with SSO.
Aster Client Guide
-p [ --port ] PORT
Aster Database server port (default: "2406").
-U [ --username ] USERNAME
Aster Database username (default: "beehive").
-w [ --password ] PASSWORD
Aster Database password. This parameter is
optional; ACT will prompt for a password if you do
not pass a -w parameter. Not used with SSO.
21
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Using the “on-error-stop” Option in ACT
The “on-error-stop” option can be used to stop ACT if an error occurs while running queries.
It discontinues executing the multi-statement SQL or the SQL file.
When the “on-error-stop” option is set, ACT halts query processing and exits as follows:
•
If ACT is in interactive mode, it returns to the ACT prompt when meeting an error.
•
If ACT is executed in the command line, it exits with status “3” when meeting an error.
Note! Unless the “-f” or “-c” options are used, ACT is always in interactive mode once it
is launched. The “\set” command can be used at the command line at any time. For more
details on the \set syntax for the “on-error-stop” option, see “Database Parameters Set
with \set” on page 35.
Setting “on-error-stop” in the command line
To enable the “on-error-stop” option (by default this option is disabled) from the command
prompt, use either of the following parameters:
• --on-error-stop
• -E
For example, to enable the “on-error-stop” in a running session of ACT, type the following
at the ACT prompt:
beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
-E -f test_queries.sql
beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
--on-error-stop -f test_queries.sql
Use a Configuration File to Pass ACT Startup Parameters
Specifying all the necessary startup parameters for ACT on the command line can become
cumbersome, especially when using SSL. Because of this, Teradata Aster has provided a way to
specify startup parameters in a configuration file, to streamline starting ACT. On Windows
and Linux/UNIX-based operating systems, ACT looks for the configuration file in a specific
location and loads it automatically. Alternatively, the configuration file may be invoked using
the --config-file parameter when launching ACT.
All of the startup parameters in ACT are supported in the config file, except for the following:
•
--config-file
•
-V [ --version ]
•
-? [ --help ]
To use a configuration file, first create a text file of startup parameters. The following rules
apply when creating the config file:
1
Lines starting with a # character are ignored (considered as comments).
2
Blank lines are ignored (including lines containing just spaces).
3
Parameters are entered using the format
flagname: value
22
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
where flagname is same as the name of the command line flag without the preceding
hyphens (--) and value is the flag value as it would be provided on the command line.
Note that the short notations of flags are not supported. For example:
host: <ip>
will work but the following:
h: <ip>
will not work.
4
Flags which do not take any argument on the command line should be given a value of
either true or false.
5
Flag names are case-sensitive.
6
If the config file includes invalid flag names or repeated entries, ACT will not launch, and
an error will display.
7
If the config file includes the “on-error-stop” option with the parameters set to enable
this option, ACT will stop if an error occurs while running SQL queries. See Set “on-errorstop” in the ACT config file (page 23) for information on setting this option.
Set “on-error-stop” in the ACT config file
To enable the “on-error-stop” option, add it to the ACT config file: "actconfig.ini" and set
the value to "true". Set the value to "false" to turn off this option. By default, this feature is
disabled or “off ”.
The following is an example from of a config file for ACT:
# ACT configuration file example
# Contains setting to enable on-error-stop
host: 10.10.10.10
dbname: sampledb
username: beehive
on-error-stop: true
The default location for the config file on Linux/UNIX-based systems is $(HOME)/
.actconfig. For Windows, the default location for the config file is %HOMEPATH%/
actconfig.ini. Upon launching, ACT will look for the config file in the default location,
and if found, will use it by default.
Command line flags can be specified when starting ACT, in addition to the flags in the
configuration file. If a flag is present in the config file and specified at the command line as
well, the value on the command line will override the value in the config file. ACT will notify
the user that the command line flag was used upon startup.
The following is an example of a config file for ACT:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
Aster Client Guide
23
Aster Database Cluster Terminal (ACT)
How to Use ACT
enable-ssl: true
ssl-self-signed-peer: true
ssl-encrypt-reads: false
To start ACT, explicitly using the config file, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config
To start ACT, explicitly using the config file and also specifying an additional parameter to
redirect query results to a file for this session only, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config -o /home/beehive/
query_results_file
How to Use ACT
Issue SQL Queries
To use ACT to run a SQL query against Aster Database, you can do one of the following:
•
Enter the query at the ACT command line
•
Run the query from a file
However, before you can run queries, you must first launch ACT, as described in “Launch
ACT” on page 16.
For example, to run queries against the retail_sales database, run this command on the Queen
to launch ACT:
[root@localhost ~]# act -d retail_sales –U beehive –w beehive
Welcome to act AC 5.10, the Aster nCluster Terminal.
Type:
\copyright for distribution terms
\h for help with SQL commands
\? for help with act commands
\g or terminate with semicolon to execute query
\q to quit
retail_sales=>
To list the tables in the database, enter \d at the ACT prompt (in this case, retail_sales=>).
For example:
retail_sales=> \d
List of relations
Schema |
Name
| Type | Owner
--------+--------------+-------+--------public | customer_dim | table | beehive
public | date_dim
| table | beehive
public | geo_dim
| table | beehive
public | product_dim | table | beehive
public | region_dim
| table | beehive
public | sales_fact
| table | beehive
public | store_dim
| table | beehive
(7 rows)
24
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
Run queries from the command line
To control how ACT handles errors when running single or multi-line SQL queries from the
command line, see “Using the “on-error-stop” Option in ACT” on page 22.
To run a single-line SQL query from the command line:
1
Type the query at the ACT prompt.
Note! The query must end with a semicolon.
2
Press Enter.
For example, to list all the rows of the customer_dim table of the retail_sales database and
order the results based on gender, enter this command at the ACT command-line prompt:
retail_sales=> select customer_id, gender, city_id from customer_dim
order by gender;
customer_id | gender | city_id
-------------+--------+--------743 | F
|
46
2711 | F
|
124
744 | F
|
66
To run a multi-line SQL query from the command line:
1
Type the first line of the query, then press Enter.
The ACT prompt changes from => to -> (for example, retail_sales->).
2
Enter the remaining lines.
You can press Ctrl-C at any time to abandon this command mode without executing the
query.
3
On the last line, type a semicolon at the end of the line, then press Enter to run the query.
For example:
retail_sales=> select customer_id, gender, city_id
retail_sales->from customer_dim
retail_sales->order by gender;
or
retail_sales=> select customer_id, gender, city_id
retail_sales->from customer_dim
retail_sales->order by gender
retail_sales->;
Run queries from files
To control how ACT handles errors when running SQL queries from a file, see “Use a
Configuration File to Pass ACT Startup Parameters” on page 22.
To run a query stored in a file:
1
Make sure the filename ends with .sql.
The query does not have to end with a semicolon.
2
At the prompt, enter this command:
\i filename
Aster Client Guide
25
Aster Database Cluster Terminal (ACT)
How to Use ACT
For example, consider this query stored in the file myQuery.sql:
SELECT *
FROM customer_dim
ORDER BY gender
To run this query, enter this command at the ACT command prompt:
retail_sales=> \i myQuery.sql
You can also run this query using the –f flag of the act command. For example:
[root@localhost ~]# act -d retail_sales -w beehive -f myQuery.sql
Workaround for Multibyte Characters
ACT does not accept multibyte characters from the SQL prompt. For example, the following
statement will produce a syntax error:
beehive=> create database db
The workaround to input multibyte characters is as follows:
1
2
Input the multibyte characters from an SQL file instead of at the SQL prompt:
a
Create a file containing the SQL to insert the multibyte data (for example,
insert_file.sql).
b
Execute it by issuing \i insert_file.sql.
Or, use the SQL command buffer in ACT:
a
Input the data using the query buffer, by issuing the ACT \e command, which brings
up the default text editor.
b
Type in the SQL, including the multibyte characters.
c
Execute the SQL by exiting the text editor (for example by typing :wq if your default
text editor is vi.)
Exit ACT
To quit ACT, type \q and hit <Enter>.
Page Through Query Results
When ACT returns results to the screen, it (by default) prints one page of results at a time. Hit
the space bar to display the next page of results, hit <Enter> to display just one more line of
results, and type “q” to quit looking at results and return to the SQL prompt. When you get to
the last page of results, ACT displays the text, “(END)”. Type “q” to return to the SQL prompt.
Throttle Query Results in ACT and Aster Database
To reduce the memory footprint of ACT and other Aster Database clients, you can set a fetchcount that constrains the number of rows returned at one time when you select from a large
table. You can limit the total number of rows returned by using fetch-limit. Additionally, you
can make your query results stream more efficiently by using the server-side cursors feature. In
26
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
ACT, you can set these parameters using \set, and in other clients you can typically set them
in your data source definition or parameters file.
Let’s look at fetch-count first.
Reduce Memory Use With Fetch-Count
The purpose of using fetch-count is to reduce the memory footprint of ACT on the server.
Therefore, this type of configuration would normally be done by an administrator, or at least
the setting would be made under an administrator’s guidance. To set a fetch-count, you
specify a value for the fetch-count parameter in ACT. This causes ACT to fetch rows in sets,
with each set containing only the user-specified number of rows (or fewer rows).
ACT uses a fetch count by default, i.e. even when fetch-count is not set explicitly. The
fetch-count (number of rows per fetch) should always be set to greater than 0, The default
value is 1024 rows.
To set the fetch-count in a running session of ACT, use the \set command to set the
fetch-count parameter. To do this, type the following at the ACT prompt:
\set fetch-count n
where n is the maximum number of rows ACT should return at a time.
To enforce the fetch-count, ACT uses server side cursors to fetch results, which can help
prevent the memory footprint of ACT from growing too large.
Note that to the user, the results returned will not be different when using fetch-count. The
purpose is simply to reduce the memory footprint of ACT on the server.
Limit Caching With Server-Side Cursors
Setting use-server-cursors has the same effect as declaring a server-side cursor with the
CURSOR syntax in SQL. When server-side cursors are activated, one batch of data is retrieved
at a time. This lets the queen avoid caching the entire result set and lets the workers continue
performing computations while the queen is retrieving rows. As the result set becomes
available from a worker, the queen sends it to the client.
Depending on the type of query, results can start streaming back to the client immediately
(e.g. SELECT * WHERE...) or the results may have to be computed in their entirety (e.g.
GROUP BY) before streaming to the client. Regardless of whether the results begin to stream
to the client before the query has finished executing, server-side cursors can improve
performance significantly for large result sets. If you want to start streaming results of a query
as soon as possible, you can benefit from setting use-server-cursors and specifying a
fetch-count. By default, use-server-cursors is set to 0 (off).
Example of Fetch-Count With Server-Side Cursors
The following steps are an example of how to use server-side cursors and fetch-count together:
1
Enable server cursors by issuing the following command inside ACT (a setting of 1 turns
cursors “on” and a setting of 0 means “off ”):
beehive=> \set use-server-cursors 1
2
Aster Client Guide
Set a fetch-count of 100:
27
Aster Database Cluster Terminal (ACT)
How to Use ACT
beehive=> \set fetch-count 100
3
Issue the following SQL query:
SELECT A,B,COUNT(*) FROM lineitem GROUP BY a, b;
This will actually cause the following statements to be run:
BEGIN;
DECLARE x
FETCH 100
FETCH 100
...
...
FETCH 100
CLOSE x;
END;
CURSOR FOR SELECT A,B,COUNT(*) FROM LINEITEM GROUP BY A,B;
FROM x; // 100
FROM x; // 200
FROM x; // until all the rows have been fetched
Aster Database reports all the fetched rows as the row count for this query.
Set the Maximum Number of Rows Returned With Fetch-Limit
To limit the total number of rows returned by each query, set the fetch-limit in ACT. fetchlimit (the total number of rows fetched for a query) can be set to any value. A value less than
0 implies fetch all rows. A value greater than 0 implies fetch-limit rows in total. The default
value is -1 (all rows).
Tip! When fetch-limit is used, the total row count returned for the query will be the total row count returned
by the query or the row count specified by fetch-limit, whichever is smaller. For example, if a query normally
returns 35,453 rows, but you have specified a fetch limit of 1000, the query will return 1000 rows (and it will display
“1000 rows returned”). There will be no indication that there were in fact 35,453 rows that would have been returned
had you not had a fetch limit set.
In ACT, you set the fetch-limit by typing:
\set fetch-limit n
where n is the maximum number of rows a single query will be allowed to return.
The SQL LIMIT Clause vs. Fetch-Limit and Fetch-Count
Setting a fetch-limit is an alternative to specifying a LIMIT clause explicitly in your
SELECT query, with an important difference in how the query will be executed, which can
affect performance. fetch-limit is an ACT feature which controls how many rows are
fetched by the queen from the workers and by the client from the queen. ACT stops fetching
results from the cursor after the specified limit is reached. LIMIT is a SQL clause which
controls how many rows are computed on the worker(s) or the queen.
The following examples illustrate the difference between using fetch-limit and using a
LIMIT clause in the SQL query:
Consider the statement:
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Example 1: Using fetch-limit and fetch-count
To issue the query in ACT with a fetch-count of 100 and a fetch-limit of 1000:
28
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
\set fetch-count 100
\set fetch-limit 1000
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Here is how the query will execute:
1
ACT opens a cursor on the query.
2
The queen activates Parallel Cursors and passes the query
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
to the workers.
3
Each worker then computes the entire equi-join as dictated by the FOO.A = BAR.A
constraint.
4
The queen fetches results from workers in batches of 100 rows, until a limit of 1000 is
reached.
5
ACT fetches results from the queen in batches of 100 rows, until a limit of 1000 is reached.
Example 2: Using a LIMIT clause in the SQL query
To issue the same query in ACT using the SQL LIMIT clause to limit results to 1000 rows:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
Here is how the query will execute:
1
ACT issues the SQL statement to the queen.
2
The queen planner identifies LIMIT 1000 and pushes down the following query to each of
the workers:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
3
Each worker computes the equi-join as dictated by the FOO.A = BAR.A LIMIT 1000
constraint. This allows for optimizations where the entire equi-join computation need not
be done at each worker.
4
The queen fetches results from the workers.
5
ACT fetches results from the queen.
The difference between the two approaches occurs in step 3 of both examples. Using the SQL
LIMIT clause (Example 2), the workers are allowed to compute with the constraint of LIMIT
1000, whereas in Example 1 they have to compute the entire equi-join.
So, as you can see, fetch-limit via server-side cursors does not translate into the workers
doing a LIMIT 1000 on their individual slice of data. Therefore, if the use case calls for it, an
Aster Database power-user should be aware that using the SQL LIMIT clause can speed up
query execution dramatically in Aster Database.
ACT Utility Commands
ACT also offers many utility commands that provide assistance and carry out useful utility
operations such as uploading files and changing output options. These commands start with a
backslash character, and we describe them in the section, “ACT Commands (at the SQL
Prompt)” on page 30.
Aster Client Guide
29
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Repeat Previously Typed Commands
Hit the up arrow on your keyboard to toggle through your history of previously typed
commands. To edit a previously typed command, just use the usual combination left arrow,
right arrow, delete, and backspace keys to make your edits, and type <Enter> to finish editing
the line. If needed, type “;” to issue the command. To list your command history on Linux,
type \s.
Tip! If you receive an “Error writing history to file.” error on Linux when attempting to view command history with \s,
check that the current Linux user has permissions to write to the current working directory.
Tab Completion
The UNIX/Linux version of ACT can tab-complete SQL commands and table names that you
type. Tab-completion is not available in the Windows version of ACT.
To use this feature, type the first couple of letters of a command and hit the <Tab> key. If the
completion is unambiguous, ACT completes the command. If ACT doesn’t complete the
command, hit <Tab> again and ACT prints all the possible completions. Using the list as a
reference, type enough additional characters to unambiguously identify the desired command
or table, and hit <Tab> again to complete it. Here are a few common uses of tab completion:
•
To complete common SQL commands. For example, type “se” and hit <Tab> to type
SELECT.
•
To list various ACT utility commands. For example, type “\”and hit <Tab> to show all the
commands, or type “\d” and hit <Tab> to show all the commands that start with “d”.
•
To complete a table name. For example, type “SELECT * FROM sa” and hit <Tab> to
complete the table name or hit <Tab> twice show all the table names that start with “sa”.
You can also list the names of all tables in the database by typing “SELECT * FROM ” (note
the trailing space) and hitting <Tab> twice.
ACT Commands (at the SQL Prompt)
The ACT SQL prompt accepts a number of ACT-specific commands that you issue by typing a
backslash followed by a character or combination of characters and arguments. Do not type a
semicolon to conclude these commands! Hitting <Enter> executes the command.
Note! Don’t confuse these with the ACT command-line startup parameters (also known as
“command line flags”), which you type at the shell command line when you launch ACT. For
a list of those, see “Startup Parameters for ACT” on page 18.
We describe the SQL-prompt commands in the tables that follow:
30
•
General-purpose utility commands in ACT (page 31)
•
Environment settings in ACT (page 31)
•
Query Buffer Commands in ACT (page 32)
•
Input/output-related commands in ACT (page 32)
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
•
Installed-function and installed-file management commands in ACT (page 32)
•
Informational commands in ACT (page 33)
•
Formatting-related commands in ACT (page 34)
Table 2 - 7: General-purpose utility commands in ACT
Command
Description
\?
prints help for ACT commands.
\c[onnect]
HOST PORT
\c[onnect]
HOST
\c[onnect]
\c[onnect]
DBNAME USER
DBNAME USER
DBNAME USER
DBNAME
change login credentials and/or connect to a new database.
The parameters must be specified in the order shown, with a
space before each, and parameters may not be skipped. In
other words, if only one parameter is specified, it is
understood to be DBNAME; if a second parameter is also
specified, it is understood to be USER; and so on.
\cd [DIR]
change the current working directory.
\copyright
show ACT usage and distribution terms.
\h
help with SQL commands.
\h [SQL command name]
help with syntax of the specified SQL command, * for all
commands.
\g
or terminate with a semicolon (;) to execute query.
\q
quit ACT.
\! [command]
execute command in shell or start interactive shell.
\password
change the password for the current user.
Table 2 - 8: Environment settings in ACT
Aster Client Guide
Command
Description
\info
display current environment settings.
\set
display current ACT parameter settings.
\set param-name
[param-value]
set ACT parameter setting param-name to value paramvalue. (For example, “\set fetch-count 500” tells ACT
to fetch no more than 500 rows at a time when selecting.) If no
parameter value is supplied, displays the current setting for the
specified parameter.
\timing [on|off]
toggle or set timing of commands.
\pager [on|off]
toggle or set to use pager to enable paging through large result
sets.
31
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 2 - 9: Query Buffer Commands in ACT
Command
Description
\e [FILE]
edit the query buffer (or file) with external editor. On most systems, this
launches your default text editor. When you save and exit the editor, the
edited statement is passed back to ACT for running.
\g [FILE]
send query buffer to server (and results to file or | (pipe character)).
\p
show the contents of the query buffer.
\r
reset (clear) the query buffer.
\w FILE
write query buffer to file.
Table 2 - 10: Input/output-related commands in ACT
Command
Description
\echo
[STRING]
write string to query output stream (see \o below).
\i [FILE]
execute SQL commands from SQL script file. (Run an SQL script.)
\o [FILE]
redirect all query results to file or | (pipe character).
\o
Type \o with no argument to stop sending results to a file and resume sending
them to the ACT shell.
\s
[FILENAME]
display command history in Linux (optionally, print history to a file specified by
FILENAME) Note that query history includes only the first 2048 characters of
each query.
Table 2 - 11: Installed-function and installed-file management commands in ACT
Command
Description
\dF
list installed files, SQL-MapReduce functions, and other functions in the
current schema. Use a regular expression as an argument to display a subset of
the available functions. For example, to view all installed functions in the
database, issue:
\dF *.*
where the first asterisk means "all schemas" and the second means "all
functions and files."
Alternatively, you can examine the system views.
\dF+
show details for all installed files, SQL-MapReduce functions, and other
functions in the current schema. For each function, the output shows the
name, schema, owner, upload time, and MD5 Hash fingerprint of the
function.
Use a regular expression as an argument to display a subset of the available
functions. For Example, type \dF+ *.* to show details for functions and
files in all schemas in the database. Alternatively, you can examine the system
views.
32
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 2 - 11: Installed-function and installed-file management commands in ACT (continued)
Command
Description
\dE
show all the installed SQL-MR functions for which the current user has
privileges. Use a regular expression as an argument to display a subset of the
available functions. Shows function name, schema, owner, function version
and creation time.
\install
<FILE>
[[<SCHEMA>/
]<FILE_ALIAS>
]
install the file or SQL-MapReduce function in Aster Database. The file must
be available on the file system where ACT is running. Note that the database
user running this command must have permission to install files and
functions in the specified schema.
You cannot install two files or functions with the same name. If attempting to
do this, you must follow these steps:
• remove the existing file or function
• install the new file or function
• grant the appropriate privileges on the file or function.
There is a limit of 238MB on the size of the file to be installed. If you try to
install a larger file, you will see an error like:
ERROR: row text exceeds limit of 238MB ...
Note that when installing larger files, the queen may run out of memory. The
queen needs available memory of approximately eight times the size of the file
to be installed, in order to encode, buffer and copy the file.
\download
[[<SCHEMA>/
]<FILE_ALIAS>
] <FILE>
download the specified installed file or function (identified by its FILE or
FILE_ALIAS) to the machine where ACT is running. Note that the database
user running this command must have permission to download files and
functions from the specified schema.
\remove
[[<SCHEMA>/
]<FILE_ALIAS>
]
remove from the cluster the file or SQL-MapReduce function specified by its
FILE_ALIAS. Note that the database user running this command must have
permission to remove files and functions from the specified schema.
Table 2 - 12: Informational commands in ACT
Aster Client Guide
Command
Description
\d
list all tables, indexes and views in the current schema.
\d [PATTERN}
describe table or index.
\dt
list all tables in the current schema.
\dt [PATTERN}
print schema, name, type, and owner of a table or
tables. To see tables in a custom schema, type \dt
schemaname.*
\dv
list views.
\dv [PATTERN]
describe view.
\di
list indexes.
33
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 2 - 12: Informational commands in ACT (continued)
Command
Description
\di [PATTERN}
describe index.
\dg
list groups.
\dg [PATTERN}
describe group.
\du
list users.
\du [PATTERN}
describe user.
\dn
list schemas.
\dn [PATTERN}
describe schema.
\l
list all databases.
\extl host=hostname_or_IP
[option_name=option_value,
…]
lists all databases on an external system.
\extd host=hostname_or_IP
database=dbname
[option_name=option_value,
…]
lists all tables in a database or describe a table on an
external system.
Tip! In Aster Database 5.0, for the \extd command in ACT, if the optional user argument is not specified, the
command will fail on any but the default database. The error message is not specific about what caused the command
to fail. The workaround is to always specify the argument user when issuing \extd.
Tip! ACT uses the schema search path (search_path) for the database user when displaying lists of tables,
veiws and indexes. The schema search path defaults to the schema search path for the current user in the database.
To set the search_path from ACT, issue the following command:
beehive=> SET session search_path TO <schema>;
Note that multiple schemas are not supported. If multiple schemas are listed in the search_path, the first
schema listed will be used.
To display the current search_path type:
beehive=> SHOW search_path;
Note that you may also set the search_path on the server.
Alternatively, you can specify the schema to use when issuing commands by following the command with a schema
qualified reference. This example shows how to display information on all tables in the schema “myschema”:
\dt myschema.*
Table 2 - 13: Formatting-related commands in ACT
34
Command
Description
\a
toggle between unaligned and aligned output mode.
\f [STRING]
show or set field separator for unaligned query output.
\t [on|off]
show only rows (off by default).
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Database Parameters Set with \set
Table 2 - 13: Formatting-related commands in ACT (continued)
Command
Description
\x [on|off]
set or toggle expanded output mode ON and OFF. With expanded output
mode turned on, each record is split into rows, with one row for each
value, and each new record is introduced with a text label in the form, --[ RECORD 37 ]---. This can help make wide tables readable on a
small screen, and is very useful if you’re trying to read EXPLAIN output.
Note that in expanded mode, the number of rows is not returned at the
end of the table. Because of this, when querying a table with no rows,
you will simply see the ACT prompt again.
Database Parameters Set with \set
The syntax to use the \set command is:
\set [ name [ value [ ... ] ] ]
Table 2 - 14: Database parameters
Parameter
Description
auto-commit [1|0]
When set to 1 (the default, on), each SQL command is automatically
committed upon successful completion. When set to 0 (off), you may
manually commit your changes after each transaction or series of
transactions by issuing the COMMIT command, or undo changes by
issuing ROLLBACK. If you do not issue the COMMIT command, all
transactions that occurred since the last COMMIT will rollback
automatically.
fetch-count [int]
To limit the number of rows returned at a time. ACT uses a fetch count
by default (i.e. even when fetch-count is not set explicitly.) The
fetch-count (number of rows per fetch) should always be set to
greater than 0, The default value is 1024 (1024 rows).
fetch-limit [int]
To set the maximum number of rows returned per query. A value less
than 0 implies fetch all rows. A value greater than 0 implies fetchlimit rows in total. The default value is -1 (all rows).
use-server-cursors
[1|0]
When set to 1, sets the server to use cursors (useful when the result set
is very large). When set to 0 (the default, off), sets the server to not use
cursors.
on-error-stop [1|0]
By default, this feature is disabled or set to off = 0.
When set to 1 (or “on”) ACT will stop and exit if it meets an error
during SQL query processing.
The following are ACT exit messages:
• EXIT_SUCCESS = 0 means ACT finished processing normally.
• EXIT_FAILURE = 1 means an error occurred, such as "file not
found" in the “-f ” option.
• EXIT_USER = 3 means an error occurred in a sql script and the
option “on-error-stop” was on or enabled.
Aster Client Guide
35
Aster Database Cluster Terminal (ACT)
Common SSL Configurations
Common SSL Configurations
This section presents common SSL configuration scenarios and shows how to set up each one
when using ACT. For more general information on setting up SSL, see SSL Security for Aster
Database-Client Communications (page 77).
In addition to the server settings, the command line argument --enable-ssl must be used
when invoking ACT. If you attempt to use an SSL argument (--ssl-self-signed-peer, for
example) without --enable-ssl, a warning message will display. In addition,
secureMuleServer=true must be set on the server if any SSL command line arguments are
used in ACT.
The common SSL scenarios are:
•
Scenario 1: Queen Provides a Self-Signed Certificate (page 37)
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate (page 38)
•
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate (page 39)
•
Scenario 4: Encrypting Communication from the Queen to the Client (page 40)
Setting Parameters on the Queen
To set up any of the following SSL configurations, the parameter secureMuleServer=true
must be set on the queen. In addition, each scenario includes some additional SSL settings
that must be made on the queen. See How to Set Configuration Parameters on the Queen
(page 85) for information on how to set these parameters.
Encrypting Communications from the Queen
By default, communications from the client to the queen (for example, queries submitted to
the database) are SSL encrypted, and query results travel in the clear. To encrypt query results,
follow the steps in Scenario 4: Encrypting Communication from the Queen to the Client in
addition to the steps to set up SSL.
Using Single Sign-on (SSO)
For information on setting up SSO on the queen, see Adding AD-Based SSO Authentication
to SSL-Secured Aster Database (page 84). Note that when using SSO, the -U and -w options
are not used, because the username and password are passed directly to the host via SSO. Also,
when using SSO, you should specify a fully qualified hostname using the -h option, as in the
example: <hostname>.<domain>.<com|org etc>. If only the hostname is used, ACT will
append the local domain name before attempting to look up the host. Using an IP address
with -h is not supported with SSO.
The following is an example of a configuration file that uses SSO:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: saturn.asterdata.com
36
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Common SSL Configurations
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-self-signed-peer: true
# SSO settings
enable-sso: true
Scenario 1: Queen Provides a Self-Signed Certificate
In this scenario, the server provides a self-signed certificate to the client at the time of
authentication. The certificate on the queen will be signed by the server itself, as opposed to by
a CA (certification authority). In such a configuration, any client can establish an SSL-secured
connection, provided the user authenticates successfully with Aster Database.
Queen-Side Settings
Make the following settings on the queen (note that these are the default settings for a new
Aster Database installation):
•
disallowPeerWithoutCertificates=false
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
Ensure that secureWrites is set to false
•
Ensure that secureMuleServer is set to true
•
There is no need to set the trustedCAPath and trustedCAFileName parameters.
Client-Side Settings
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-self-signed-peer
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-self-signed-peer: true
Note that the Client need not have a copy of the server's certificate. Do not use the other SSL
settings such as --ssl-trusted-ca-file, or --ssl-trusted-ca-path.
Aster Client Guide
37
Aster Database Cluster Terminal (ACT)
Common SSL Configurations
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate
In this scenario, ACT (the client) needs a CA-signed copy of the queen’s certificate. The
location of this certificate can be specified with --ssl-trusted-ca-file, or as a chain of
trusted certificates using the --ssl-trusted-ca-dir parameter. For more specific
information on using a certificate chain, see Queen-Side SSL Parameters (page 79).
Queen-Side Settings
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=false
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
•
Ensure that secureWrites is set to false.
•
Ensure that secureMuleServer is set to true.
•
There is no need to set the trustedCAPath and trustedCAFileName parameters.
Client-Side Settings
Do the following:
Copy the queen's public key (self-signed certificate), /home/beehive/certs/server.pem,
to the client. For this example, we will assume the client will store the public key as /home/
jbloggs/certs/server.pem.
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-trusted-ca-file server.pem
•
--ssl-trusted-ca-dir /home/jbloggs/certs/
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-trusted-ca-file: server.pem
ssl-trusted-ca-dir: /home/jbloggs/certs/
Because --ssl-self-signed-peer is not specified, the connection can be made only when
the server provides a CA signed certificate. The identical CA signed certificate must already
exist on the client. However, when --ssl-self-signed-peer is used, the server is able to
supply the certificate at the time of connection and nothing is required on the client.
38
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Common SSL Configurations
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate
This scenario presents a stricter regime, where the queen only accepts connections from
clients that provide a CA-signed certificate, for which a copy already exists on the queen at the
time of connection. In other words, clients cannot connect using a self-signed certificate, nor
can they connect using a copy of the queen's public key. The identical signed certificate must
exist on the queen as well. Setting the server flag disallowPeerWithoutCertificates =
true is what forces the client to provide a certificate in order to connect.
Queen-Side Settings
Do the following:
1
Get the root certificate of the CA (certificate authority) that signed your client's certificate.
Save the root certificate on the queen. For this example, we will save it as /home/
beehive/certs/client.pem on the queen.
2
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
trustedCAFileName=/home/beehive/certs/client.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
•
There is no need to set the trustedCAPath parameter if you use a single root
certificate for all clients.
•
Ensure that secureWrites is set to false.
•
Ensure that secureMuleServer is set to true.
Variation: If your client's certificates were not all signed by the same CA, then you must set
Aster Database to recognize all the CA root certificates used to sign you clients' certificates,
like so:
1
Save the root certificates of all the signing CAs on the queen.
2
Set trustedCAPath to point to the directory that contains the root certificates. For
example:
•
3
trustedCAPath=/home/beehive/certs
Un-set the queen configuration parameter, trustedCAFileName, by setting it to no value
at all. For example:
•
trustedCAFileName=
Client-Side Settings
Use the following command line arguments when executing ACT. For this example, we will
assume the client will store the certificate as /home/jbloggs/certs/client.cert and the
key as /home/jbloggs/certs/client.key:
•
Aster Client Guide
--enable-ssl
39
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
•
--ssl-certificate-path /home/jbloggs/certs/client.cert
•
--ssl-private-key-path /home/jbloggs/certs/client.key
•
--ssl-cert-filetype 1 (A value of "1" means SSL_FILETYPE_PEM. A value of
“2” means SSL_FILETYPE_ASN1.)
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-certificate-path: /home/jbloggs/certs/client.cert
ssl-private-key-path: /home/jbloggs/certs/client.key
ssl-cert-filetype: 1
Scenario 4: Encrypting Communication from the Queen to the Client
By default, only the queries issued by ACT to the queen are encrypted. It is also possible to
encrypt the communication from the queen to ACT. Note that these changes can be applied to
all of the previous scenarios. To do this, make the following changes in addition to the settings
necessary to enable SSL.
Queen-Side Settings
Make the following setting on the queen:
•
secureWrites=true
Even though you may have set secureWrites=false when setting up SSL, set it to true now
in order to enable encryption of communication from the queen. This permits setting up the
SSL connection before enabling two way encryption, if desired.
Client-Side Settings
Use the following command line arguments when executing ACT:
•
--ssl-encrypt-reads
Or add the following entry to the config file:
•
ssl-encrypt-reads: 1
Troubleshooting ACT
ACT Connection Hangs When Using SSL
When ACT is configured to use SSL, the queen must also be set up to use SSL. If the queen is
not setup for SSL and ACT tries to connect using SSL, the connection will fail. An error is not
40
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
returned; the ACT client just hangs. If you experience this, check to ensure that SSL settings
on the queen and in ACT match.
Misleading Error Message Reports Problem With a Role Instead of With a
User
When attempting to connect to a database as a user who does not have connect privileges on
that database, the error incorrectly reports that the problem is with a role instead of with the
user. This example illustrates the error message:
beehive=> create user kris with password 'beehive';
CREATE USER
beehive=> create database foo;
CREATE DATABASE
beehive=> \c foo kris;
Password for "kris":
act: ERROR: role "kris" does not have connect privileges on this
database.
Previous connection kept
Aster Client Guide
41
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
42
Aster Client Guide
Connect Using Database Drivers
CHAPTER 3
Aster Database provides Open Database Connectivity (ODBC) and Java Database
Connectivity (JDBC) drivers for connecting business intelligence (BI) tools. This section
explains how to install and use the Aster Database drivers.
General tips:
•
General Tips for Connecting Clients to Aster Database (page 44)
Setting up database drivers:
•
ODBC Driver (page 44)
•
JDBC Driver (page 59)
•
Tools for .NET Environments (page 97): ADO.NET driver (also known as Aster Database
DNProvider driver) for SSIS, SSAS, SSRS, and .NET
Using database drivers:
•
Process SQL Statements in JDBC (page 90)
•
JDBC Troubleshooting and Limitations (page 92)
•
Configure Aster Database SQL Settings (page 71)
Security for database drivers:
•
Teradata Wallet (page 73)
•
SSL Security for Aster Database-Client Communications (page 77)
Connecting BI and query tools:
•
Aster Client Guide
Connect Reporting Tools to Aster Database (page 92)
43
Connect Using Database Drivers
General Tips for Connecting Clients to Aster Database
General Tips for Connecting Clients to Aster
Database
Recommended Character Set Is UTF-8
For all tools that you use to connect to databases in Aster, you will typically want to set the
default character set to UTF-8. This is particularly important if you plan to use special
characters (for example, German letters ä and ß) in a char, varchar, or text column, and it is
particularly important if you are connecting from a Windows-based machine.
For example, if you will use an SSH client (for example, putty) to run ACT or ncluster_loader,
make sure you set the SSH client’s default character set to UTF-8.
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'
Cursors cannot be declared and used with Aster Database system tables. The default mode of
autocommit-on with the ODBC driver declares cursors. Therefore, the ODBC driver should
always query system tables by turning AUTOCOMMIT to 'off '.
AIX Client Dependent Libaries
For Aster AIX Clients, the following dependent libraries are required.
Table 3 - 1: Dependent LIbraries for AIX Clients
Required Library
ODBC
Aster Loader and Export Tools
libstdc++.a (gcc)
4.6.1
4.6.1
libgcc_s.a (gcc)
4.6.1
4.6.1
ODBC Driver
Teradata Aster provides a standard ODBC driver for Aster Database. The following list shows
all the supported operating systems with the corresponding ODBC driver package name:
44
•
Windows 32-bit: nClusterODBCInstaller_i386.msi
•
Windows 64-bit: nClusterODBCInstaller_x64.msi
•
Linux 32-bit: clients-odbc-linux32.tar.gz
•
Linux 64-bit: clients-odbc-linux64.tar.gz
•
Solaris Sparc 32: clients-odbc-solaris-sparc.tar.gz
•
Solaris i386: clients-odbc-solaris-x86.tar.gz
•
Mac OS x86: clients-odbc-mac.tar.gz
•
AIX Power PC 32-bit: clients-odbc-aix.tar.gz
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
•
AIX Power PC 64-bit: clients-odbc-aix.tar.gz
The Aster Database ODBC driver may change in any Aster Database release. For this reason, with each new edition of
Aster Database, you should reinstall the driver and recompile your applications that include the driver.
ODBC Driver Manager Compatibility
The Aster Database ODBC driver requires a driver manager. Different platforms use different
driver managers. Here is a list of platforms and their supported ODBC driver managers:
•
Windows: Microsoft Driver Manager which is part of the Windows operating system.
•
Linux: Driver Manager, which is bundled and automatically installed with the Aster
Database ODBC driver.
•
AIX: Driver Manager, which is bundled and automatically installed with the Aster
Database ODBC driver.
•
Solaris: unixODBC driver manager
•
MacOS: iODBC driver manager
The following diagram illustrates the path the ODBC connection takes from an application to
Aster Database:
Figure 1: Path of an ODBC connection
Linux/AIX
ODBC
Application
Driver Manager
Supplied by Aster Database
Aster Database
ODBC Driver
Aster
Database
Driver Manager
Supplied by Microsoft
Aster Database
ODBC Driver
Aster
Database
unixODBC Driver Manager
(Open Source)
Aster Database
ODBC Driver
Aster
Database
iODBC Driver Manager
(Open Source)
Aster Database
ODBC Driver
Aster
Database
Windows
ODBC
Application
Solaris
ODBC
Application
Mac OS X
ODBC
Application
Obtain the ODBC Driver Package
First, you’ll need to get the appropriate Aster Database ODBC driver package for your
operating system. To obtain the package, download it from one of these places onto your
client machine in the directory where you will install it:
Aster Client Guide
•
To get the newest package, download it from http://downloads.teradata.com/download/tools
•
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
45
Connect Using Database Drivers
ODBC Driver
Next, see the section below that applies to your platform:
•
Install ODBC on Windows (page 46)
•
Install ODBC on Linux, Solaris, or MacOS (page 48)
•
Install ODBC on AIX (page 50)
Install ODBC on Windows
Follow the steps below to install the Aster Database ODBC driver on Windows.
Driver Manager Support
Windows includes Driver Manager, which allows ODBC applications to use the correct driver.
Installation Procedure
1
Verify that Microsoft Visual C++ 2008 Redistributable Package (x86) is installed on the
system where you want to install the driver. Note that Microsoft also offers newer versions
of the package, such as Microsoft Visual C++ 2010 Redistributable Package (x86). Teradata
Aster has not tested compatibility with these later versions! If the supported version is not
installed:
a
Download it from Microsoft, choosing the version that fits your architecture:
•
32-bit
•
64-bit
2
Obtain the ODBC Driver Package.
3
If you are upgrading, use the Windows Add/Remove Programs tool to uninstall the old Aster
ODBC version.
4
Double-click on the .msi file to install it.
A setup wizard walks you through the installation of Aster Database ODBC as one of the
available data sources on your computer.
46
5
From the Windows Control Panel, double-click Administrative Tools to open the
Administrative Tools window.
6
Double-click the Data Sources (ODBC) option to open the ODBC Data Source
Administrator dialog box.
7
Click the System DSN tab.
8
Click Add to open the Create New Data Source dialog box.
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
9
Select the Aster ODBC Driver data source from the list.
10 Click Finish.
The Aster Database Login window appears.
11 In the Aster Database Login window, enter the following information:
•
Data Source: Use this field to give this database connection an easy-to-recognize name.
•
Server: The hostname or IP address of your Aster Database queen.
•
Port: The port on which your Aster Database queen listens for client connections. The
default is 2406.
•
Database: The name of the database in Aster Database you want to connect to. Default
system database is beehive.
Aster Client Guide
•
Username: Database user name.
•
Password: Database user’s password.
•
Fetch Count: See “Throttle Query Results in ACT and Aster Database” on page 26.
47
Connect Using Database Drivers
ODBC Driver
•
SSL Settings: See “SSL Security for Aster Database-Client Communications” on
page 77.
•
SSO Settings: See “Adding AD-Based SSO Authentication to SSL-Secured Aster
Database” on page 84.
•
enable_quoted_identifiers: See “Quoted-Identifier Handling” on page 71.
•
enable_backslash_escapes: See “Escape Character Handling” on page 71.
•
Map NUMERIC/DECIMAL to DOUBLE: Teradata Aster recommends that you turn this
feature on.
•
Bytea As Varchar: The login window does not provide a check box for specifying that
values in a column of datatype bytea should be retrieved in a character representation
(as opposed to the default binary representation). However, you can set this later in the
Windows registry. See “Optional ODBC Setting for bytea-Stored Data” on page 57.
12 Click OK to save the data source information.
13 Click Test Connection to test the connectivity to the database. If the connection is successful,
the Aster Database Login window closes automatically.
14 If this window does not close automatically, click OK.
Your ODBC setup is complete. Now, in your applications that will query Aster Database, you
may connect to Aster Database as an ODBC data source.
Install ODBC on Linux, Solaris, or MacOS
To install the Aster Database ODBC driver on Linux, Solaris, or MacOS, follow the
instructions below.
Driver Manager Support
The Aster Database ODBC driver for Linux comes with Driver Manager, which is installed
automatically when you install the Aster Database ODBC driver. Driver Manager allows
ODBC applications to use the correct driver (in this case, the Aster Database ODBC driver) to
connect to Aster Database.
The Driver Manager bundled with Aster Database ODBC supports these Linux platforms:
•
Linux 32-bit
•
Linux 64-bit
For Solaris, use unixODBC driver manager and for MacOS, use iODBC manager.
Prerequisites
1
48
The Aster Database ODBC driver requires the following libgcc version, depending on your
operating system:
•
Linux and Solaris: libgcc 3.4.6
•
Mac OS 10.5: libgcc 4.2
•
Mac OS 10.6: libgcc 4.5
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
2
On Solaris and MacOS systems, the Aster Database ODBC driver requires that you install a
driver manager:
•
On Solaris, use the unixODBC driver manager, version 2.2.12. Instructions are
available here: “Install and Configure the unixODBC Driver Manager on Solaris” on
page 52.
•
On MacOS, use the iODBC driver manager, version 3.52.3. This driver and its
documentation are available from http://www.iodbc.org.
Installation Procedure
Install the Aster Database ODBC driver as shown below.
Warning! For MacOS, the Aster Database ODBC Driver supports only MacOS version 10.6 or earlier.
1
Extract the ODBC package into the directory where you want to install the driver. Once
extracted, you will see the directory stage that includes the driver.
2
Change to the Aster Database driver directory:
# cd stage/clients-odbc-<your_client_os>
3
Edit your library path environment variable, to add the Aster ODBC library directory to it.
The Aster ODBC library directory has the path <install location>/stage/clientsodbc-<your_client_os>/Libs.
The library path variable is:
•
LD_LIBRARY_PATH on Linux and Solaris
•
DYLD_LIBRARY_PATH on MacOS
Edit the appropriate environment settings file to do this (for example, edit the ~/.bashrc
file if you want to set it for the current user on a typical Linux environment). To set it for
the current session only, type the command shown below, substituting
DYLD_LIBRARY_PATH for LD_LIBRARY_PATH on MacOS:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/stage/clientsodbc-<your_client_os>/Libs
4
Add or edit the ODBCSYSINI environment variable, setting it to the directory where your
ODBC connection settings files (odbc.ini and odbcinst.ini) will reside. To follow this
example, let’s assume we are working as user “mjones” and will save the configuration files
to our home directory /home/mjones.
export ODBCSYSINI=/home/mjones
5
Check that the Aster Database ODBC driver library can find all its dependencies.
Assuming we have installed in /usr/local/lib, we would type (on Linux or Solaris):
# cd /usr/local/lib
# ldd stage/clients-odbc-linux64/ODBCDriver/
libAsterDriver_unixODBC.so
On MacOS, we would type:
# cd /usr/local/lib
Aster Client Guide
49
Connect Using Database Drivers
ODBC Driver
# otool -L stage/clients-odbc-linux64/ODBCDriver/
libAsterDriver_unixODBC.dylib
If a “not found” message does not appear, then all the required libraries have been linked.
6
Choose the next step, depending on your operating system:
•
Configure Driver Manager on Linux and AIX
•
Install and Configure the unixODBC Driver Manager on Solaris
•
For MacOS, go to http://www.iodbc.org to download and configure iODBC driver
manager.
Install ODBC on AIX
Aster Database ODBC driver for AIX supports these platforms:
•
AIX 5.3 (32-bit and 64-bit)
•
AIX 6.1 (32-bit and 64-bit)
See also “AIX Client Dependent Libaries” on page 44.
Driver Manager Support
The Aster Database ODBC driver for AIX comes with Driver Manager, which is installed
automatically when you install the Aster Database ODBC driver. No other ODBC driver
manager is supported on AIX. Driver Manager allows ODBC applications to use the correct
driver (in this case, the Aster Database ODBC driver) to connect to Aster Database.
Installation Procedure
Install the Aster Database ODBC driver on AIX as shown below.
1
Obtain the ODBC Driver Package.
2
Extract the bundle for the ODBC driver:
a
Unzip the file:
$ gunzip clients-odbc-aix.tar.gz
b
Untar the file:
$ tar -xvf clients-odbc-aix.tar
Next Step: Configure Driver Manager on Linux and AIX.
Configure Driver Manager on Linux and AIX
Use the following process to configure the Driver Manager that is installed with the Aster
ODBC Driver on Linux and AIX:
1
Find the Driver Manager configuration files under the extracted directory:
$ cd stage/clients-odbc-<your_client_os>/DataDirect/setup
2
Copy the odbcinst.ini, odbc.ini, and aster.ini files from the /setup directory to your home
directory:
$ cp -p * ~
3
50
Change to your home directory:
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
$ cd
4
Make backups of the files you moved:
$ cp -p aster.ini aster.ini.backup
$ cp -p odbc.ini odbc.ini.backup
$ cp -p odbcinst.ini odbcinst.ini.backup
5
Make the following edits to aster.ini:
a
Set DriverManagerEncoding to UTF-8.
b
Set ODBCInstLib to <InstallDir>/DataDirect/lib/odbcinst.so, replacing
<InstallDir> with the folder where the driver is installed.
For example:
[driver]
DriverManagerEncoding=UTF-8
ODBCInstLib = <InstallDir>/DataDirect/lib/odbcinst.so
ErrorMessagesPath=<InstallDir>/ErrorMessages
DSILogging=0
6
Modify odbc.ini as follows:
a
Change the DSN configuration parameters SERVER, UID, PWD, DATABASE and
PORT.
[ODBC Data Sources]
... ...
asterdsn=AsterDriver
[ODBC]
... ...
[asterdsn]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
SERVER=192.206.82.100
PORT=2406
DATABASE=beehive
UID=beehive
PWD=beehive
b
Add this item to the [ODBC] section of odbc.ini:
InstallDir=<InstallDir>/DataDirect
7
8
Add the <InstallDir>/Libs directory to:
•
LD_LIBRARY_PATH for Linux, or
•
LIBPATH for AIX PowerPC.
Export the following values, where <directory_path> is the path to the directory where the
files odbc.ini and odbcinst.ini reside:
export ODBCHOME=<directory_path>
export ODBCINI=$ODBCHOME/odbc.ini
export ODBCINST=$ODBCHOME/odbcinst.ini
9
Edit the odbcinst.ini file, as shown in this example:
[ODBC Drivers]
... ...
AsterDriver=Installed
[ODBC Translators]
OEM to ANSI=Installed
[ODBC]
Aster Client Guide
51
Connect Using Database Drivers
ODBC Driver
#This section must contain values for DSN-less connections
#if no odbc.ini file exists. If an odbc.ini file exists,
#the values from that [ODBC] section are used.
[AsterDriver]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
IconvEncoding=UCS-4LE
Install and Configure the unixODBC Driver Manager on Solaris
On Solaris, Teradata supports using the unixODBC driver manager. This section describes
how to install and configure unixODBC.
Install the unixODBC driver manager
If you already have the unixODBC manager installed, skip this section and proceed to
“Configure the unixODBC driver manager” on page 52.
1
Download the unixODBC driver manager v 2.2.12, from http://www.unixodbc.org/
unixODBC-2.2.12.tar.gz
2
Install unixODBC by running the following commands:
# CFLAGS="-DSIZEOF_LONG=8" ./configure --prefix=/usr \
--sysconfdir=/etc/unixodbc \
--enable-fdb \
--disable-gui &&
make
3
Wait while it builds and installs.
4
Working as root user, run this:
# make install &&
find doc -name "Makefile*" -exec rm {} \; &&
chmod 644 doc/{lst,ProgrammerManual/Tutorial}/* &&
install -v -m755 -d /usr/share/doc/unixODBC-2.2.12 &&
cp -v -R doc/* /usr/share/doc/unixODBC-2.2.12
5
Check the installation by typing the following command. This prints version information
and lists the names and locations of the configuration files you will need to install or set
up.
# odbcinst -j
If your unixODBC installation is not working properly, check the instructions at http://
for help.
www.unixodbc.org/
Next Step: Proceed to the next section, Install and Configure the unixODBC Driver
Manager on Solaris.
Configure the unixODBC driver manager
If you have chosen to use the unixODBC Driver Manager instead of the bundled Driver
Manager, configure it as follows:
52
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
1
Get the templates for the ODBC connection settings files. Copy these files from the Aster
Database driver’s Setup directory to the user’s home directory. The files you need are
aster.ini, odbc.ini and odbcinst.ini:
#
#
#
#
cd
cp
cp
cp
/usr/local/lib/stage/clients-odbc-linux64/Setup
odbc.ini ~
odbcinst.ini ~
aster.ini ~/.aster.ini
Note that we have also renamed the aster.ini file, adding a dot at the beginning of the
file name. You must do this.
2
Edit the .aster.ini file as follows:
a
Set DriverManagerEncoding to UTF-32.
b
Set ODBCInstLib to <unixODBCDir>/lib/libodbcinst.so, replacing
<unixODBCDir> with the directory where the driver is installed:
c
Set the ErrorMessagesPath to point to the ErrorMessages subdirectory in the Aster
Database driver directory. For this example, we edit the last line in the file to read:
ErrorMessagesPath=/usr/local/lib/stage/clients-odbc-linux64/
ErrorMessages
Tip! At this point, you can run “odbcinst -j” to find out where the ODBC driver expects to find its configuration files.
3
In a text editor, edit the odbc.ini file, making the following changes:
a
Set SERVER to the hostname or IP address of your Aster Database queen.
b
Set PORT to 2406, the standard port on which your Aster Database queen listens for
client connections.
c
Set DATABASE to the name of the database in Aster Database you want to connect to.
d
Optionally, you may set UID and PWD to your Aster Database SQL username and
password, respectively.
e
Finally, Teradata Aster recommends that you add the setting,
NumericAndDecimalAsDouble=1.
f
If you retrieve bytea-stored data through the ODBC driver, you can specify whether
values in a column of datatype bytea will be retrieved in a character representation, or
in the default binary representation. To have the ODBC driver to retrieve values in
character representation, add the setting, ByteaAsVarchar=1 to your odbc.ini; if
you leave it unset, the driver preserves the binary output representation of bytea data.
g
Optionally, you can set a number of other database connection behavior settings.
These include enable_quoted_identifiers (see “Quoted-Identifier Handling” on
page 71), enable_backslash_escapes: See “Escape Character Handling” on
page 71.
For this example, we set the contents of odbc.ini to read:
[ODBC Data Sources]
Aster Data ODBC for nCluster DSN=AsterDriver
Aster Client Guide
53
Connect Using Database Drivers
ODBC Driver
[Aster Data ODBC for nCluster DSN]
Driver=AsterDriver
SERVER=10.50.52.100
PORT=2406
DATABASE=beehive
NumericAndDecimalAsDouble=1
# UID=<USERNAME>
# PWD=<PASSWORD>
Tip! You can have multiple data sources. The name “Aster Data ODBC for nCluster DSN” in the odbc.ini file is just
a default name that Teradata Aster has given to the sample data source. You can rename this source and add more, as
shown in this example:
[ODBC Data Sources]
my_1st_source=AsterDriver
my_2nd_source=AsterDriver
[my_1st_source]
Driver=AsterDriver
SERVER=10.50.52.100
...
[my_2nd_source]
Driver=AsterDriver
SERVER=10.42.43.100
...
4
In a text editor, edit the odbcinst.ini file, setting the Driver parameter to the Aster
Database driver directory path. For this example, we set the contents of odbcinst.ini to
read:
[AsterDriver]
Driver=/usr/local/lib/stage/clients-odbc-linux64/ODBCDriver/
libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
On MacOS, it will look like:
[AsterDriver]
Driver=/usr/local/lib/stage/clients-odbc-linux64/ODBCDriver/
libAsterDriver_unixODBC.dylib
IconvEncoding=UCS-4LE
5
The installation and configuration are now complete.
Test the unixODBC Driver Manager
Once the driver is installed and configured, you should test it:
The unixODBC driver manager comes with a tool called “isql” that you can use to test the
driver. You can test your connection now by connecting to Aster Database using the isql tool.
1
Type the command as shown below. Here, we put the data source name in single quotes
because it contains spaces. The “beehive” and “beehive” are the default username and
password in a new Aster Database installation:
# isql -v 'Aster Data ODBC for nCluster DSN' beehive beehive
54
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
2
A “Connected!” message indicates that the driver is working correctly:
+---------------------------------------+
| Connected!
|
|
|
| sql-statement
|
| help [tablename]
|
| quit
|
|
|
+---------------------------------------+
SQL>
3
Set-up is complete.
Troubleshooting
If after installation you cannot connect:
1
Find the library libodbcinst.so and note its path.
2
Set the LD_LIBRARY_PATH environment variable so that it includes the directory that
contains libodbcinst.so. For example, if the library is in /usr/lib64, then you will
type:
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
Set Up ODBC for Perl Connectivity on Linux
To install the Aster Database ODBC driver and configure your environment to allow Perl
scripts to connect to the database through the driver, follow these steps. These instructions
assume you have installed Perl on your workstation, and that the Aster Database queen is
reachable on the network. Follow these steps to provide an Aster Database connection your
Perl scripts can use:
1
Set up your /etc/resolv.conf and /etc/nsswitch.conf for accessing the Aster
Database queen.
2
Install and Aster Database ODBC. Follow the instructions in “Install ODBC on Linux,
Solaris, or MacOS” on page 48.
3
Make sure that Driver Manager is properly configured. Follow the instructions in
“Configure Driver Manager on Linux and AIX” on page 50.
4
The server connection information is set in the odbc.ini file, as explained earlier in this
chapter. On Windows, the server connection information is set using the ODBC Data
Source Administrator tool as explained in “Set up ODBC for PHP” on page 56.
5
In /root type
$ perl -eshell -MCPAN
6
Get the latest packages and install them by entering:
$ install Bundle::CPAN
$ install DBI
7
Rebuild and install DBD::ODBC.
$ export LD_LIBRARY_PATH=/home/beehive/toolchain/x86_64-unknownlinux-gnu/unixODBC-2.2.12/lib
Aster Client Guide
55
Connect Using Database Drivers
ODBC Driver
$ export PATH=$PATH:/home/beehive/toolchain/x86_64-unknown-linuxgnu/unixODBC-2.2.12/bin
$ which odbc_config
/home/beehive/toolchain/x86_64-unknown-linux-gnu/unixODBC-2.2.12/
bin/odbc_config
$ odbc_config --cflags -DHAVE_UNISTD_H -DHAVE_PWD_H
-DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG=8
$ perl -eshell -MCPAN
cpan[1]> force install DBD::ODBC
8
Run odbcinst -j to see where the .ini files are being picked up:
$ odbcinst -j
9
Run the following Perl script to check your installation.
If everything is set correctly, it will run without an error. If you encounter problems, check
the data source name (“DSN”) in the connect statement. In the example below, the
username and password of the database are “beehive” and “beehive”:
#!/usr/bin/perl
use DBI;
use DBD::ODBC;
# Connect to ODBC DSN and turn off AutoCommit
my $dbh = DBI->connect('dbi:ODBC:nc',"beehive","beehive",
{AutoCommit => 0});
$dbh->do("BEGIN");
$dbh->do("set random_page_cost to '4'");
$dbh->do("set enable_seqscan to 'off'");
$dbh->disconnect;
Set up ODBC for PHP
To set up PHP to work with ODBC, first follow all of the instructions to set up ODBC on
Linux: “Install ODBC on Linux, Solaris, or MacOS” on page 48.
As described in the instructions, you should use unixODBC 2.2.12 and build it with the
CFLAG, SIZEOF_LONG=8. The same will apply to building PHP when compiled to be used
with unixODBC. Here are the supported versions:
•
PHP 5.4.10
•
Apache 2.2.23
PHP
To set up PHP:
1
Make sure Apache is installed and make note of the directory. For this example, we will
assume Apache is installed at /usr/local/apache
2
Ensure that unixODBC has been installed as described above.
3
Download the source for PHP 5.4.10 and extract it to the desired directory. The following
setup instructions should be used for PHP:
$ CFLAGS="-DSIZEOF_LONG=8" ./configure --with-apxs2=/usr/local/
apache/bin/apxs --with-zlib --with-unixODBC --with-pdo-odbc=unixODBC
56
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
$ make && make install
$ cp -p .libs/libphp5.so /usr/local/apache/modules
Apache and PHP
Apart from the typical PHP-Apache setup, there are two things to keep in mind:
1
Ensure that the Linux system is set up to find the libraries that the ODBC driver requires.
a
Add the <INSTALL-DIR>/Libs and <INSTALL-DIR>/ODBCDriver paths to the /etc/
ld.so.conf file, where <INSTALL-DIR> is the location for the ODBC installation.
b
The following commands should be executed to refresh the ld cache and restart the
Apache web server:
$/sbin/ldconfig
$/etc/init.d/apachectl restart
2
To make sure that the ODBCSYSINI environment variable is available when PHP calls are
made, use the following line in your PHP code:
#given ODBCSYSINI is to be set to /etc then:
putenv("ODBCSYSINI=/etc")
Tip! You should not set up your own PHP or use /etc/init.d/apachectl on the queen for your own web
pages.
ODBC Usage Notes
Optional ODBC Setting for bytea-Stored Data
If you retrieve bytea-stored data through the ODBC driver, you can specify whether values in a
column of datatype bytea will be retrieved in a character representation, or in the default
binary representation.
To set this up you must set the ByteaAsVarchar setting. Setting ByteaAsVarchar=1 instructs
the ODBC driver to retrieve values in character representation; leaving it unset preserves the
binary output representation.
•
To set this up on Linux, Solaris, or MacOS, see the discussion of the odbc.ini file in
“Configure the unixODBC driver manager” on page 52.
•
To set this up on Windows, see below.
Bytea-Stored Data Settings on Windows
See the section that applies to your operating system and driver:
•
For a 64-bit ODBC driver running on 64-bit Windows, or for a 32-bit driver on 32-bit
Windows:
Set the flag by adding it to the Windows registry entry for the DSN. Using a registry editor,
add this line to the registry, taking care to first replace <DSN-NAME> with your Data
Source name:
Aster Client Guide
57
Connect Using Database Drivers
ODBC Driver
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\<DSN-NAME>]
"ByteaAsVarchar"="1"
•
For a 32-bit ODBC driver running on 64-bit Windows
Set the flag by adding it to the Windows registry entry for the DSN. Using a registry editor,
add this line to the registry, taking care to first replace <DSN-NAME> with your Data
Source name:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\<DSN-name>]
"ByteaAsVarchar"="1"
Avoid using bind_param with SQL_DECIMAL
When using the bind_param method call with decimal fields, you should not explicitly specify
SQL_DECIMAL as the parameter bind type. If you do this, and the parameter value has a
fractional digital part, the factional part changes to 0 after applying bind_param. The
DBD::ODBC bind_param implementation makes this conversion when specifying
SQL_DECIMAL/SQL_NUMERIC as the parameter bind type. Note that bind_param works
fine when SQL_DECIMAL is not explicitly specified, so you should avoid this when passing
decimal fields.
Bind columns without specifying a SQL datatype
Use the Perl API without specifying a SQL datatype the Aster datatypes, as in the example:
$rc = $sth->bind_col($column_number, \$var_to_bind);
Avoid specifying the SQL datatype to bind to the column, as in:
$rc = $sth->bind_col($column_number, \$var_to_bind, $bind_type);
This will work fine when binding boolean and character columns with DBI::SQL_CHAR, but
for the other datatypes, this will cause two issues:
1
If you specify a numeric SQL type in bind_col, corrupt data may be returned. Numeric
types include SQL_INTEGER, SQL_SMALLINT, SQL_DOUBLE, SQL_REAL,
SQL_NUMERIC/SQL_DECIMAL, SQL_BIT, SQL_TIME, SQL_TIMESTAMP,
SQL_DATE, and SQL_DATETIME.
2
If you specify a SQL_VARCHAR type in bind_col, an error may be returned. The error
looks like:
•
On Unix ODBC: “DBD::ODBC::st fetch failed: [unixODBC][Driver Manager]Invalid
application buffer type (SQL-HY003)”
•
On Windows ODBC: “DBD::ODBC::st fetch failed: [Microsoft][ODBC Driver
Manager] Program type out of range (SQL-HY003)”
This issue has been observed with the following software versions:
58
•
Perl 5.8.9
•
DBD-ODBC 1.3.1
•
DBI-1.616
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
JDBC Driver
JDBC is an API for the Java programming language that provides methods for querying and
updating data in a relational database. The Aster Database JDBC driver enables your Java
applications and reporting tools to retrieve data directly from Aster Databases.
The Aster Database JDBC driver is a Type 4 JDBC driver that implements the JDBC 3
specification.
•
Aster JDBC Driver (page 59)
•
Differences from the Legacy JDBC Driver (page 60)
•
Before You Start (page 60)
•
Install the JDBC Driver (page 61)
•
Use the JDBC Driver in a Java Application (page 61)
•
Parameters for Connecting through JDBC (page 62)
•
Configuring the JDBC Log Settings (page 63)
•
Behavior and Performance Settings for JDBC (page 63)
•
Using Client-Side Cursors in JDBC (page 67)
•
Test JDBC Connect Program (page 69)
Aster JDBC Driver
This version of the JDBC driver, introduced in Aster Client release 5.0.3, adds these
capabilities to those that were already supported in the legacy JDBC driver:
Aster Client Guide
•
Ability to handle larger data sets than the legacy JDBC driver; doesn’t try to load whole
result set onto the client in all cases
•
Support for multibyte characters in data and user metadata UTF-8 encoding
•
Support for Single Sign-On (SSO) authentication
•
Support for Secure Socket Layers (SSL) encrypted communications
•
Support customized connection parameters
•
Support configuring statement FetchSize to limit the data being returned
•
Support server side cursors
•
Support specifying maximum number of rows
•
Support for these commands:
•
Copy—Moves data between Aster Database tables and a remote client (from and to a
file) via the connection between the client and the server.
•
Install File—Installs the data file or SQL-MapReduce function in the specified Aster
Database schema.
•
Uninstall File—Removes the file or SQL-MapReduce function from Aster Database.
•
Download File—Downloads the specified installed file or function.
59
Connect Using Database Drivers
JDBC Driver
Differences from the Legacy JDBC Driver
These are important differences between this JDBC driver and the legacy (pre-5.0.3) JDBC
driver:
•
JDBC only supports scrollable forward cursors. Only the con.createStatement
(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY) and
connection.createStatement() functions are supported. The Other parameter values like
ResultSet.TYPE_SCROLL_INSENSITIVE and ResultSet.CONCUR_UPDATABLE are not
supported.
•
The executeBatch() function throws a SQL exception (java.sql.SQLException).
•
The result set cannot be accessed after an explicit commit.
•
After running PreparedStatement.executenatch(), you must set all bind values or else the
driver throws an error.
•
The java.sql.PreparedStatement.setObject() function does not throw exceptions for invalid
java.sql.Types.TINYINT values. Instead, the function wraps the values around. This
behavior complies with the Java standard.
•
ResultSetMetaData cannot be accessed after a ResultSet reset.
•
The driver cannot establish a connection to the database when there are white space
characters to the left of the connection string.
•
The register serverConnect to database using JDBC in Aqua Data Studio (ADS) as register
server. Expand the tree view when connection is lost. Register server will prompt an error.
•
The DatabaseMetaData.getBestRowIdentifier() function is not supported.
•
The ResultSet.getArray() function is not supported.
•
You can call com.asterdata.ncluster.Driver.ASTER_BUILD_VERSION() to get JDBC
version information. For detailed version information, check the MANIFEST.MF file.
Before You Start
The SSO support in the JDBC driver is based on Quest Authentication service. You must
acquire the SSO license from Quest in order to take advantage of SSO support in this client.
This support is for connecting to Aster Database only, and not for connecting through the
Teradata Connector or SQL-H.
Prerequisites
The JDBC driver supports these versions of the Java JDK:
60
•
Oracle JDK 1.5
•
IBM JDK 1.6
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Install the JDBC Driver
Follow these steps to install the Aster Database JDBC driver:
1
Obtain the package that contains the Aster Database JDBC driver . You can do this in one
of these ways:
•
To get the newest package, download it from http://downloads.teradata.com/download/
tools
•
2
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
Unzip the ZIP package.
The resulting folder contains multiple JAR files.
3
Copy the JAR files to a location in the classpath of the application that uses the driver.
Use the JDBC Driver in a Java Application
Follow these steps to integrate the Aster Database JDBC driver into your Java application:
1
Set the CLASSPATH environment variable to include the paths of the JAR files extracted
from the Aster Database JDBC driver package.
•
Use the Java -classpath flag at the command line to add the paths to the CLASSPATH
environment variable. For example, if you place the JAR files in /usr/jars, then add the
path like this:
java -classpath /usr/my_program /usr/jars/
or
•
Add the paths to the driver to the CLASSPATH environment variable. For example:
export CLASSPATH=/usr/jars/:$CLASSPATH
2
In your application:
a
Import the java.sql package like this:
import java.sql.*;
b
Load the driver using the Class.forName() method:
Class.forName("com.asterdata.ncluster.Driver");
c
Define the username, password, and url (including the host, port, and database)
parameters, which are needed to connect to the database. For example:
String username = "user";
String password = "password";
String url
= "jdbc:ncluster://myhost:2406/database";
For more information about the URL format, see “Required Parameters” on page 62.
d
Get a Connection instance from JDBC using the DriverManager.getConnection()
method:
try
{
Connection conn =
DriverManager.getConnection(url, username, password);
}
Aster Client Guide
61
Connect Using Database Drivers
JDBC Driver
catch (SQLException ex)
{
// could not connect
}
NOTE: The getConnection method throws a “No driver available SQLException”
exception if CLASSPATH does not contain the path for the JDBC driver, or if the
parameters are incorrect.
Parameters for Connecting through JDBC
Required Parameters
To establish a connection to an Aster Database using the Aster Database JDBC driver, you
must provide the driver with the URL to use to connect to the database. The URL has this
format:
jdbc:ncluster://<Host:Port>/<Database>?enable_backslash_escapes=<on_or
off>&enable_quoted_identifiers=<on_or off>
For example:
jdbc:ncluster://192.65.197.90:2406/
beehive?enable_backslash_escapes=on&enable_quoted_identifiers=on
The URL needs three parameters to connect to an Aster Database:
Table 3 - 2: Parameters in URL to connect to an Aster Database
Parameter
Required? Description
Host
Optional
The name of the server where the database resides.
To specify an IPv6 address, enclose the this parameter
in square brackets. For example:
jdbc:ncluster://[::1]:2406/nCluster
Default is localhost.
Port
Optional
The port number that the database server is listening
on.
Default is 2406.
Database
Required
The database name.
enable_backslash_escapes
Optional
Sets the queen parameter enable_backslash_escapes
for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
enable_quoted_identifiers
Optional
Sets the queen parameter enable_quoted_identifiers
for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
In addition, to the URL, you must also provide the username and password needed to access
the Aster Database, which you can get from your Aster Database administrator.
62
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Optional Parameters
You can set the Autocommit and fetch_count settings for the connection in the URL by adding
the autocommit and fetch_count parameters. See “Frequently Used JDBC Settings” on
page 63. If your application will query large tables, you should set autocommit to false, and
you should declare a fetch_count for the connection. By doing this, you enable the
connection to use distributed cursors for improved performance.
You can also use the NumericAndDecimalAsDouble parameter to map NUMERIC and DECIMAL
type columns to SQL_DOUBLE. When you set this parameter, its value will be stored in a connection
context. This example shows how to use this parameter to decode the row header:
if ((sqlType == Types.NUMERIC ||
sqlType == Types.DECIMAL) &&
inSettings.connectionSettings_.numericAsDouble_) {
sqlType = Types.DOUBLE;
}
Configuring the JDBC Log Settings
To enable and configure JDBC logging:
1
Create a file named simba.properties and add it to the folder containing the JDBC driver.
2
In the file, define the LogLevel and LogPath properties.
•
LogLevel—(Optional) Specifies the log level: OFF, FATAL, ERROR, WARNING, INFO,
DEBUG, and TRACE.
•
LogPath—(Optional) Specifies the path to the log file (AsterJDBC.log). If this property
is not defined, the JDBC driver stores the log file in the same folder where the JDBC
driver is located.
For example:
LogLevel=DEBUG
LogPath=D:\\logs
Behavior and Performance Settings for JDBC
When using the Aster Database JDBC driver, you make most behavior and performance
settings in your SQL code using the SET command.
You can also use a number of standard JDBC driver methods for setting behavior and
performance options, such as setAutoCommit() and setPrepareThreshold().
The scope of these variables are limited to the scope of the transaction. After the commit()
call, the transaction variables revert to their original values. In order to limit the transaction
variables to a particular query, the variables need to be saved prior to being set and once the
query is executed the variables must be set back to their original values.
Frequently Used JDBC Settings
•
autocommit: Boolean value that sets the autocommit behavior for the connection, true to
turn on autocommit (each statement, by default concluded with a semicolon, is run
immediately in the database) or false for off (you must COMMIT your transaction to run
Aster Client Guide
63
Connect Using Database Drivers
JDBC Driver
it). You can set this in the connection URL with the autocommit parameter (for example,
jdbc:ncluster://10.80.50.100:2406/beehive?autocommit=false) or in the Java
code for your connection with Connection.setAutoCommit(). The default is true.
Unsupported JDBC Settings
Most JDBC option-setting methods in the Aster Database driver exhibit the standard JDBC
behavior, but there are exceptions. Please note:
•
setReadOnly(): Not supported; Aster Database does not allow changing connection type
to read-only.
•
setTransactionIsolation(): Not supported; Aster Database does not allow changing
transaction isolation levels.
Example Code Snippet That Sets Performance Tuning Variables
This example (PerformanceTuning.java) shows you how you can set some performance
tuning variables for a particular transaction:
package userguide;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class PerformanceTuning {
public static void main(String[] args) {
String username = "db_superuser";
String password = "db_superuser";
String url = "jdbc:ncluster://153.65.197.125:2406/beehive";
Connection conn = null;
try
{
conn =
DriverManager.getConnection(url, username, password);
// Remember the current autocommit state
boolean autoCommit = conn.getAutoCommit();
// Turn off auto commits
conn.setAutoCommit(false);
// create a statement with transaction variables set
// which will be used for this query
Statement stmt = conn.createStatement();
stmt.executeUpdate("SET enable_hashjoin = 'false'");
stmt.executeUpdate("SET enable_mergejoin = 'false'");
stmt.executeUpdate("SET enable_nestloop='true'");
stmt.executeUpdate("SET random_page_cost = '4.0'");
Statement statement = conn.createStatement();
String query="select * from knn_test";
ResultSet resultSet = statement.executeQuery(query);
// Process the result set
while (resultSet.next()) {
// do something
}
64
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
// commit the transaction
conn.commit();
resultSet.close();
stmt.close();
// Set the autocommit state back
conn.setAutoCommit(autoCommit);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
In the example above, the scope of the SET variables is limited to the commands between the
autoCommit(false) and commit() lines.
The Copy, Install File, Uninstall File, and Download File Commands
Version 5.0.3 of the JDBC driver adds support for these commands:
•
Copy (page 65)
•
Install File (page 65)
•
Uninstall File (page 66)
•
Download File (page 67)
For more information about these commands, see the section describing SQL commands in
the Aster Database User Guide.
Copy
This command moves data between Aster Database tables and a remote client (from and to a
file) via the connection between the client and the server.
This is an example:
public void copyCommandExamples() {
String stmt1 = "COPY simba TO 'd:\\simba.txt' DELIMITER as ','";
String stmt2 = "COPY simba TO 'd:\\simba.csv' with csv QUOTE AS '@';";
try {
Statement s = conn_.createStatement();
s.execute(stmt1);
s.execute(stmt2);
s.close();
} catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
Install File
This command installs the data file or SQL-MapReduce function in the specified Aster
Database schema.
Aster Client Guide
65
Connect Using Database Drivers
JDBC Driver
INSTALL FILE filename [[schema/]alias]
This is an example:
public void testInstallWithSchema() {
String stmt1 = "install file 'd:\\odbcng.tar.gz' ";
String stmt2 = "install file 'd:\\odbcng.tar.gz' uploadfile/'odbcng.zip'";
String stmt3 = "install file 'd:\\abc.txt' uploadfile/'abc.txt'";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
Uninstall File
This command removes the file or SQL-MapReduce function from Aster Database.
UNINSTALL FILE 'filename' from schema schemaname
This is an example:
public void testUninstall() {
String stmt1 = "UNINSTALL file 'abc.txt' from schema uploadfile ;";
String stmt2 = "UNINSTALL file 'odbcng.zip' from schema uploadfile ;";
String stmt3 = "UNINSTALL file 'odbcng.tar.gz'
;";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
66
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Download File
This command downloads the specified installed file or function.
DOWNLOAD FILE [[schema/]alias] filename
This is an example:
public void download_file_examples() {
String stmt1 = "DOWNLOAD file uploadfile/'abc.txt' 'd:\\download-abc.txt'";
String stmt2 = "DOWNLOAD file uploadfile/'odbcng.zip' 'd:\\download-uploadfileodbcng.zip'";
String stmt3 = "DOWNLOAD file 'odbcng.tar.gz' 'd:\\download-public-odbcng.zip'";
try {
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
s.close();
} catch (Exception e) {
fail(e.getMessage());
}
}
Using Client-Side Cursors in JDBC
To give you control over the latency of initial query results, the Aster Database JDBC driver
supports client-side cursors. Client-side cursors let you avoid retrieving all the result rows for
a query at once. Instead, you specify the number of rows that should be retrieved to the client
at a time. When that set is exhausted, the next page of rows is retrieved by repositioning the
cursor.
Observe the following guidelines when working with cursors:
1
Turn off autocommit mode for the Connection object. See “Using Cursors in Your Code”,
below.
2
The ResultSet object that receives the output of your Statement cannot be a scrollable
ResultSet. That is, it must have the type, ResultSet.TYPE_FORWARD_ONLY. This is the
default, so you need not rewrite your code.
3
If your application will query large tables, you should make sure your ResultSet can use
distributed cursors. To do this,
a
make sure the ResultSet is not updatable (ResultSet.CONCUR_READ_ONLY)
b
make sure it’s not scrollable (ResultSet.TYPE_FORWARD_ONLY)
c
make sure it does not have HOLD enabled
(ResultSet.CLOSE_CURSORS_AT_COMMIT).
d
Also, the application should connect with autocommit set to false and the connection
must have a specified fetch_count.
Tip! When working with ResultSets of type ResultSet.TYPE_FORWARD_ONLY, you cannot scroll backwards, nor can you jump to any location in the ResultSet other than the next row.
Aster Client Guide
67
Connect Using Database Drivers
JDBC Driver
4
In the Statement object, you must pass a single query, not multiple queries strung together
with semicolons.
5
You must set the statement’s fetch_count using the Statement.setFetchSize(int
rows) command. This instructs the driver to fetch the specified number of rows at a time
from the database. If the fetch size is not set, the driver fetches the full set of rows that
match the query.
6
To help ensure a quick response when a page of rows is exhausted, the JDBC driver, by
default, pre-fetches and caches ten pages of results from the database. A page is one
fetch_count worth of rows. You can set the number of pages to be pre-fetched, or you can
disable pre-fetching if desired.
Using Cursors in Your Code
The driver fetches fetch_count number of rows (as set with Statement rows)) from Aster
Database and exposes them to the database application as a “rows” data structure. The default
fetch_count is zero, so you must set a fetch_count if you wish to use cursors.
With ResultSet.next(), your application iterates over the rows data structure. When all
rows in the rows data structure have been processed, the next call to next() will force the
fetching a new set of fetch_count number of rows, and so on.
To use cursors, set the fetch size of your SQL statement using the setFetchSize(int rows)
method. Later, setting the fetch_count back to 0 will cause all rows to be retrieved when the
query runs (the default behavior).
Tip! In the example below, we set the Autocommit setting and the FetchSize setting using the setAutoCommit() and setFetchSize() methods, but you also have the option of setting these in the JDBC connection
parameters when you make the connection. See “Frequently Used JDBC Settings” on page 63.
Example That Uses Client-Side Cursors
Assuming a Connection conn, you can set the FetchSize like this:
// Remember the current autocommit state
boolean autoCommit = conn.getAutoCommit();
// Make sure autocommit is off
conn.setAutoCommit(false);
Statement st = conn.createStatement();
// Turn use of the cursor on by setting FetchSize, expressed in rows
st.setFetchSize(50);
ResultSet rs = st.executeQuery("SELECT * FROM mytable");
// Set the autocommit state back
conn.setAutoCommit(autoCommit);
while (rs.next()) {
System.out.print("a row was returned.");
}
rs.close();
// Close the statement.
st.close();
68
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Disabling Cursors
Cursors are enabled by default. To turn them off, set the FetchSize to zero. Assuming a
statement st and a ResultSet rs, you would do this as shown here:
st.setFetchSize(0);
rs = st.executeQuery("SELECT * FROM mytable");
while (rs.next()) {
System.out.print("many rows were returned.");
}
Test JDBC Connect Program
This program can be used as a template for your JDBC connectivity:
// JDBC Test Program
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.ResultSet;
public class jdbctest {
static String userid="beehive", password = "beehive";
// Change the IP address to the node you want to connect to.
static String url = "jdbc:ncluster://10.60.3.100:2406/beehive"; //2.0
static Connection con = null;
public static void main(String[] args) throws Exception {
Connection con = getJDBCConnection();
if(con!= null){
try {
int count = 0;
System.out.println("Got Connection.");
DatabaseMetaData meta = con.getMetaData();
System.out.println("getDriverName: "+meta.getDriverName());
System.out.println("getDriverVersion: "+meta.getDriverVersion());
System.out.println("getDriverMajorVersion:
"+meta.getDriverMajorVersion());
System.out.println("getDriverMinorVersion:
"+meta.getDriverMinorVersion());
System.out.println("getDatabaseProductName:
"+meta.getDatabaseProductName());
System.out.println("getDatabaseProductVersion:
"+meta.getDatabaseProductVersion());
System.out.println("getIdentifierQuoteString:
"+meta.getIdentifierQuoteString());
System.out.println("\ngetSQLKeywords: "+meta.getSQLKeywords());
System.out.println("\ngetNumericFunctions:
"+meta.getNumericFunctions());
System.out.println("\ngetStringFunctions
:
"+meta.getStringFunctions());
System.out.println("getSystemFunctions
:
"+meta.getSystemFunctions());
System.out.println("getTimeDateFunctions
:
"+meta.getTimeDateFunctions());
System.out.println("getSearchStringEscape
:
"+meta.getSearchStringEscape());
System.out.println("getExtraNameCharacters
:
"+meta.getExtraNameCharacters());
System.out.println("getCatalogTerm
:
"+meta.getCatalogTerm());
Aster Client Guide
69
Connect Using Database Drivers
JDBC Driver
System.out.println("getCatalogSeparator
:
"+meta.getCatalogSeparator());
System.out.println("getURL
:
"+meta.getURL());
System.out.println("getUserName
:
"+meta.getUserName());
System.out.println("getMaxCursorNameLength
:
"+meta.getMaxCursorNameLength());
System.out.println("getMaxSchemaNameLength
:
"+meta.getMaxSchemaNameLength());
System.out.println("getMaxProcedureNameLength :
"+meta.getMaxProcedureNameLength());
System.out.println("getMaxCatalogNameLength
:
"+meta.getMaxCatalogNameLength());
System.out.println("getMaxColumnsInIndex
:
"+meta.getMaxColumnsInIndex());
System.out.println("supportsSubqueriesInComparisons :
"+meta.supportsSubqueriesInComparisons());
System.out.println("getMaxConnections
:
"+meta.getMaxConnections());
System.out.println("getMaxColumnsInTable
:
"+meta.getMaxColumnsInTable());
System.out.println("isReadOnly
:
"+meta.isReadOnly());
System.out.println("\ngetCatalogs:");
ResultSet res = meta.getCatalogs();
while (res.next()) {
System.out.println(res.getString(1));
}
System.out.println("\ngetTables:");
res.close();
res = meta.getTables(null,null,"%",null);
while (res.next()) {
System.out.println(res.getString(1) +res.getString(2) +res.getString(3)
+res.getString(4));
}
System.out.println("");
res.close();
res = meta.getTableTypes();
System.out.println("\ngetTableTypes:");
while (res.next()) {
System.out.println(res.getString(1));
}
res.close();
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
con.close();
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
}else{
System.out.println("Could not Get Connection");
}
}
public static Connection getJDBCConnection(){
try {
Class.forName("com.asterdata.ncluster.Driver");
} catch(java.lang.ClassNotFoundException e) {
70
Aster Client Guide
Connect Using Database Drivers
Configure Aster Database SQL Settings
System.err.print("ClassNotFoundException: ");
System.err.println(e.getMessage());
}
try {
con = DriverManager.getConnection(url,userid, password);
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
return con;
}
}
Configure Aster Database SQL Settings
This section explains how to set parameters that customize the behavior of Aster Database
SQL.
SQL Behavior Parameters
Escape Character Handling
enable_backslash_escapes: Set to “on” (the default), Aster Database allows backslash
escape strings in your constants. The escape strings are C-like backslash escape sequences,
each introduced with a backslash character (\). Set to “off ”, Aster Database does not recognize
backslash escape strings in your constants unless the constant is prefixed with a letter E before
its opening single quote. In any constant not prefixed with the letter E, a backslash is
interpreted simply as a backslash character.
Quoted-Identifier Handling
enable_quoted_identifiers: This setting affects the way Aster Database processes strings
enclosed in double-quote characters ("..."). With enable_quoted_identifiers='on' (the
default), Aster Database follows the standard behavior of interpreting each double-quoted
string as an identifier (a column, table, schema, or function name). With
enable_quoted_identifiers='off', each double-quoted string is interpreted as a literal
string constant. Any printable character may be represented in a double-quoted string.
Setting the SQL Behavior Parameters
At the SQL Prompt
You can set these parameters from the SQL prompt by executing SET commands in the form:
SET SESSION <parameter name> = {'<param setting>'};
For example:
SET SESSION enable_backslash_escapes = {'on'|'off'};
and
SET SESSION enable_quoted_identifiers = {'on'|'off'};
Aster Client Guide
71
Connect Using Database Drivers
Configure Aster Database SQL Settings
In the Aster Database ODBC Driver
The Aster Database ODBC-NG driver allows you to set Aster Database SQL behavior
parameters like, for example, enable_backslash_escapes. The connection parameters are:
•
enable_backslash_escapes={on|off}
•
enable_quoted_identifiers={on|off}
These can be set in the DSN configuration (i.e., the odbc.ini file or Windows registry).
Alternatively, a connecting application may set these options in the connection string when
connecting without a registered DSN.
Syntax for ODBC Commands
This section lists some ODBC commands for which the syntax differs from the native Aster
Database syntax. Note that comments and parameter markers are not allowed in these
commands. However, the prepare/execute model is supported.
COPY FROM/TO
Semantics and overall flow of execution for COPY is the same as for the Aster Database Loader
Tool. See “Aster Database Loader Tool” on page 125.
Syntax
COPY table [(column list)] FROM <quoted file name> ...COPY attributes
or
COPY table [(column list)] TO <quoted file name> ...COPY attributes
The COPY command accepts a quoted file name and streams the data into or out of Aster
Database using the Aster Database Loader Tool protocol for maximum throughput.
INSTALL
The INSTALL command is similar to the ACT command “\install <FILE> [[<SCHEMA>/
]<FILE_ALIAS>]” on page 33, and supports SQL-MR security semantics.
Syntax
INSTALL FILE <quoted file name> [[<schema>/]<file alias>]
•
The schema name must be quoted if it contains spaces or mixed case.
•
The file alias must be quoted.
UNINSTALL
The UNINSTALL command supports SQL-MR security semantics.
Syntax
UNINSTALL FILE <quoted file name> [[<schema>/]<file alias>]
72
•
The schema name must be quoted if it contains spaces or mixed case.
•
The file alias must be quoted.
Aster Client Guide
Connect Using Database Drivers
Teradata Wallet
DOWNLOAD
The DOWNLOAD command is similar to the ACT command “\download [[<SCHEMA>/
]<FILE_ALIAS>] <FILE>” on page 33, and supports SQL-MR security semantics.
Syntax
DOWNLOAD FILE <quoted file name> [[<schema>/]<file alias>]
•
The schema name must be quoted if it contains spaces or mixed case.
•
The file alias must be quoted.
Teradata Wallet
Teradata Wallet (TD Wallet) is a software utility that provides the users of an Aster Database
client system with full and unrestricted access to their stored database passwords on that
system, while at the same time protecting those passwords from being exposed in scripts.
Each user on an Aster Database client has a TD Wallet. The TD Wallet securely stores the
passwords that the user adds to the wallet. However, a user cannot access the passwords stored
in the wallets of other users.
Rather than using the passwords in scripts, you can use their corresponding names defined in
your wallet.
The ACT, JDBC, and ODBC drivers support TD Wallet.
Wallet Contents
A wallet contains a set of <name, value> pairs. These pairs consist of Unicode character
sequences.
Table 3 - 3: Wallet contents
Item
Description
name
The name of this password entry. This is the name you use to reference the
password in scripts without exposing it. We recommend using meaningful
names to help you determine what the password is used for. The name is
case-sensitive.
value
The password. The password is not exposed in your scripts. Only the name of
this password is exposed.
The example in Table 3 - 4 shows entries in the TD Wallet of user client system user jdoe.
Table 3 - 4: The Teradata Database passwords of user jdoe
Aster Client Guide
Name
Value
pwd_for_beehive_db
s4t#gp6s_#4
pwd_for_customers_db
nsdho_34f
73
Connect Using Database Drivers
Teradata Wallet
Table 3 - 4: The Teradata Database passwords of user jdoe
Name
Value
pwd_for_clickstream_db
oc_m_3nd234
TD Wallet Commands
TD Wallet provides these commands:
Table 3 - 5: TD Wallet commands
Command
Description
tdwallet add name
Adds an item to the wallet. When you run this command,
tdwallet prompts you for the value component of the name/value
pair.
tdwallet addsk name
Adds a string with the specified name (saved-key).
tdwallet del name
Deletes the specified item from the wallet.
tdwallet list
Lists the contents of the wallet.
tdwallet chgpwd
Changes or establishes the password of the wallet.
tdwallet suppwd
Supplies the password of the wallet.
tdwallet forgetpwd
Forgets the password of the wallet.
tdwallet chgsavkey
Changes or establishes the saved-key.
tdwallet version
Displays the version information for tdwallet.
tdwallet help [topic] ...
Displays the help information for tdwallet.
Download TD Wallet
TD Wallet support has been added in the back-end client connector layer, both the Java and
C++ connectors. All the Aster Database clients based on the back-end client connector can use
it. To use TD Wallet in Aster Database, you must first download it from http://
downloads.teradata.com/download/tools. The supported version of TD Wallet is version
14.00. Note that this version of TD Wallet does not currently support MacOS.
Install and Configure TD Wallet on Linux
ACT
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
$ ./tdwallet add mypassword
Enter desired value for the string named "userpass":
String named "userpass" added.
3
74
Install the latest ACT for Aster Database.
Aster Client Guide
Connect Using Database Drivers
Teradata Wallet
4
Create the symbolic link “tdwalletdir” in the directory where ACT is installed.
For example, these commands add the symbolic link:
cd /home/beehive/work/multibranch/build/bin
ln -s /opt/teradata/client/tdwallet tdwalletdir
JDBC
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
$ ./tdwallet add mypassword
Enter desired value for the string named "userpass":
String named "userpass" added.
3
Get the JDBC Driver and tdwalletJNI.so.
4
Copy tdwalletJNI.so into /home/beehive/work/multibranch/builds/build-main/lib/.
$ javac -classpath /home/beehive/work/multibranch/builds/build-main/
lib/noarch-aster-jdbc-driver.jar com/test/testjdbc/tdwallet.java
$ java -classpath .:/home/beehive/work/multibranch/builds/build-main/
lib/noarch-aster-jdbc-driver.jar -Djava.library.path=/home/beehive/
work/multibranch/builds/build-main/lib/ com.test.testjdbc
You can also find the tdwalletJNI.so file in the same directory as noarch-aster-jdbcdriver.jar in these packages:
5
•
clients-platform-version-reversion.tar.gz
•
clients_all/platform...
Create the symbolic link “tdwalletdir” in the directory where tdwalletJNI.so located is
located.
For example, these commands add the symbolic link:
cd /home/beehive/work/multibranch/builds/build-main/lib/
ln -s /opt/teradata/client/tdwallet tdwalletdir
Install and Configure TD Wallet on Windows
ACT
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
E:\>tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Use the TD Wallet password instead of the real password to log in to Aster Database. For
example:
E:\asterclientWIN\win64\bin>act.exe -h 192.65.197.130 -U beehive
-w $tdwallet(mypassword)
Welcome to act 05.10.00.00, the Aster nCluster Terminal.
...
Aster Client Guide
75
Connect Using Database Drivers
Teradata Wallet
ODBC
1
Install TD Wallet.
2
Add name/value pairs to the wallet.
3
Install the latest ODBC Driver for Aster Database.
4
Use the TD Wallet password ($tdwallet(mypassword)) instead of the real password to log in
to Aster Database.
$tdwallet(mypassword)
JDBC
1
Install TD Wallet.
2
Add name/value pairs to the wallet.
E:\>tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Get the latest JDBC Driver, and make the dependent library libtdwalletJNI.dll available in
the library path.
E:\>javac -classpath e:\asterclient\lib\noarch-aster-jdbc-driver.jar
com\test\testjdbc\tdwallet.java
E:\>java -classpath e:\asterclient\lib Djava.library.path=e:\asterclient\lib com.test.testjdbc
4
Use the TD Wallet password ($tdwallet(mypassword)) instead of the real password to log in
to Aster Database.
Usage
After you install TD Wallet you can use the $tdwallet directive in place of your password. The
syntax if the directive is:
$tdwallet(name)
where name is the name of a password entry in the TD Wallet.
TD Wallet Password Processing Rules
When TD Wallet processes $tdwallet directives, it follows these rules:
1
If specified, process name as follows:
•
76
Replace “\)” with “)”
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
•
Replace “\$” with “$”
•
Replace “\\” with “\”
2
Query the corresponding current user’s wallet for an item with a name matching the result
of the processing in step 1.
3
Replace name with the value of the item found by the query in step 2.
Example of TDWallet Use in SQL-MR queries
In this query, TD Wallet replaces $tdwallet(abc) with the corresponding password stored
on the client TD Wallet.
select * from cfilter(
on (select 1)
partition by 1
database('beehive')
userid('beehive')
PASSWORD('$tdwallet(abc)')
inputtable('cfilter_test')
outputTable('cfilter_test1')
inputColumns('item')
joinColumns('tranid')
droptable('true'));
SSL Security for Aster Database-Client
Communications
Aster Database provides the option to use SSL to secure communications between Aster
Database ODBC clients and the Aster Database queen. By default, an ODBC client running on
Windows or Linux sends all of its communications to Aster Database over an SSL-encrypted
connection. By default, results and other data returned from Aster Database to the client are
NOT encrypted. This default configuration can be changed to suit your needs, as explained in
the sections that follow.
Port Number
The Aster Database queen port number for SSL connections is the same as the regular client
connection port: 2406. Port 2406 is multiplexed to support both secure sockets layer (SSL)
connections and unencrypted connections.
Aster Client Guide
77
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
Contents of This Section
This section contains:
•
SSL-Related Files and Settings (page 78)
•
SSL Settings Reference (page 79)
•
Common SSL Configuration Scenarios (page 80)
•
Encrypting Data Traffic (page 83)
•
Adding AD-Based SSO Authentication to SSL-Secured Aster Database (page 84)
•
How to Set Configuration Parameters on the Queen (page 85)
•
How to Set Configuration Parameters on the Client (page 85)
•
Creating Certificates (page 90)
SSL-Related Files and Settings
The queen has the following files related to SSL communications and set-up:
•
Private key: Default one is /home/beehive/certs/server.key
•
Self-signed PEM certificate (that is, the base64-encoded ASCII representation of the
certificate only; this is the version of the certificate that starts with "-----BEGIN
CERTIFICATE-----" and ends with "-----END CERTIFICATE-----"). The default one is /
home/beehive/certs/server.cert
•
Another copy of the self-signed certificate, also in PEM format, but this time formatted in
X.509 structure. Note! This file may be missing from your Aster Database installation. To
work around this problem, you must create the default PEM file from the server.cert
file, saving it as /home/beehive/certs/server.pem. To create it, log in to the queen as
root, change directories to /home/beehive/certs, and type:
# openssl x509 -in server.cert -text >> server.pem
•
The queen’s SSL-related settings (see “SSL Settings Reference” on page 79) in the Aster
Database queen configuration file, /home/beehive/config/procmgmtConfigs/
coordinator.cfg
On the client side, the files related to SSL and its configuration are:
78
•
Copy of the queen’s public key in PEM format. This is a copy of the queen’s server.pem.
For example, you might save it as /home/mjones/certs/server.pem
•
The client’s SSL-related settings (see “Client-Side SSL Settings” on page 79), stored:
•
for Linux, in the client’s odbc.ini file; and
•
for Windows, in the ODBC parameter fields of the registry
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
SSL Settings Reference
This section provides a reference to the available SSL settings on the Aster Database queen.
Queen-Side SSL Parameters
Below, we list the configuration flags that can be used on the queen to tune SSL behavior. Most
queen-side flags have a corresponding client-side flag. When you change a flag on one side
(client or server), you will typically have to make appropriate changes to the other side.
•
disallowPeerWithoutCertificates: If this flag is set, the client cannot communicate
with its peer (server) without a valid certificate. This flag is defaulted to FALSE.
•
allowSelfSignedPeer: If this flag is set, Aster Database allows connections from clients
with self-signed certificates. Defaults to TRUE.
•
Set either trustedCAFileName or trustedCAPath, depending on whether you have one
or many CA certificates:
•
trustedCAFileName: The pathname of the single PEM-formatted CA certificate that
Aster Database trusts. (You also have the option of trusting multiple CA certificates;
see trustedCAPath, below.) Whenever the queen gets a certificate from the peer, the
queen traverses the certificate chain to verify that the certificate specified by
trustedCAFileName is part of the chain. If so, the peer is allowed to connect.
•
trustedCAPath: Directory containing PEM-formatted CA certificates that Aster
Database trusts. The files inside this directory are looked up based on the CA subject
name hash value.
•
sslCertificatePath: SSL certificate location.
•
sslPrivateKeyPath: SSL private key location.
•
sslFileType: The formatting type of the certificate. Set this to a string value of 1 for
PEM-encoded certs (called “SSL_FILESYSTEM_PEM” on the client side) or 2 for ASN1encoded certs (called “SSL_FILETYPE_ASN1” on the client side). Default is 1.
•
secureMuleServer: If set, Aster Database will be configured to use a secure channel for
its communication. If secureMuleServer is enabled, the configuration flags
sslCertificatePath and sslPrivateKeyPath should be appropriately set.
Client-Side SSL Settings
Below, we list the configuration flags that can be used on the client side to tune SSL behavior.
These settings work with the Aster Database ODBC client. Most client-side flags have a
corresponding queen-side flag. When you change a flag on one side (client or server), you will
typically have to make appropriate changes to the other side.
•
•
Aster Client Guide
EnableSSL: Enables/disables the use of SSL (string value of 0 for false, or 1 for true).
•
0 = Disable SSL (default)
•
1 = Enable SSL
SSLEncryptReads: Determines whether the client expects data returned from database to
be encrypted (string value of 0 for false, or 1 for true).
•
0 = query results are unencrypted (the default)
•
1 = query results are encrypted
79
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
•
SSLAllowSelfSignedPeer: Determines whether the client allows peers with self signed
certificates to communicate (string value of 0 for false, or 1 for true; default is 1).
•
SSLFileType: The certificate file type. A string value; one of:
•
SSL_FILETYPE_PEM (the default)
•
SSL_FILETYPE_ASN1
•
SSLPrivateKeyPath: Path to the private key to be used. Optional. (A string value.)
•
SSLCertificatePath: Path to the SSL certificate to be used. (A string value.)
•
Set either SSLTrustedCADir or SSLTrustedCAFilename, depending on whether you
have one or many CA certificates:
•
SSLTrustedCADir: Path to the directory containing CA certificates in PEM format.
(A string value.)
•
SSLTrustedCAFilename: Filename of CA certificate in PEM format. (A string value.)
Common SSL Configuration Scenarios
This section presents common SSL configuration scenarios and shows how to set up each one.
The scenarios are:
•
Scenario 1: Allowing connections from clients without certificates (page 80)
•
Scenario 2: Client has a copy of the queen’s public key (page 81)
•
Scenario 3: Client has a copy of a certificate you provide (page 82)
•
Scenario 4: Client must present a valid, CA-signed certificate (page 82)
In all of the scenarios, we instruct you to set parameters on the queen and on the client. For
instructions, see:
•
For the queen: “How to Set Configuration Parameters on the Queen” on page 85; and
•
For clients: “How to Set Configuration Parameters on the Client” on page 85
Scenario 1: Allowing connections from clients without certificates
In this scenario, the Aster Database SSL configuration is edited to allow connections from
clients that have no certificate. In such a configuration, any Aster Database ODBC client can
establish an SSL-secured connection, provided the user authenticates successfully with Aster
Database. Communications from the client to Aster Database (for example, queries submitted
to the database) are SSL encrypted, and query results travel in the clear.
Queen-Side Settings
Make the following settings on the queen:
80
•
disallowPeerWithoutCertificates=false
•
allowSelfSignedPeer=true
•
trustedCAFileName=/home/beehive/certs/server.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
•
Ensure that secureWrites is set to false
•
Ensure that secureMuleServer is set to true
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
There is no need to set the other SSL settings such as SSLPrivateKeyPath.
Scenario 2: Client has a copy of the queen’s public key
In this scenario, we edit Aster Database’s SSL configuration to allow connections only from
clients that have a copy of queen’s public key. This scenario uses the default public key
(/home/beehive/certs/server.pem) that is part of the standard queen installation.
Queen-Side Settings
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/server.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
Ensure that secureWrites is set to false.
•
Ensure that secureMuleServer is set to true.
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
Do the following:
Aster Client Guide
1
Copy the queen’s public key (self-signed certificate), /home/beehive/certs/
server.pem, to the client. For this example, we will assume the client will store the public
key as /home/jbloggs/certs/server.pem.
2
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/server.pem
81
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
Scenario 3: Client has a copy of a certificate you provide
This scenario is the same as the preceding one, except that now we replace the default ASter
Database certificate with your site’s own certificate. As before, this configuration allows
connections only from clients that have a copy of queen’s public key. This scenario uses the
public key, /home/beehive/certs/sampleco.pem.
Queen-Side Settings
Do the following:
1
Get your public key file and save it on the queen. For this example, we will save it as
/home/beehive/certs/sampleco.pem on the queen.
2
Save the corresponding private key file on the queen. For this example, we will save it as
/home/beehive/certs/sampleco.key on the queen.
3
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/sampleco.pem
•
sslCertificatePath=/home/beehive/certs/sampleco.pem
•
sslPrivateKeyPath=/home/beehive/certs/sampleco.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
Do the following:
1
Copy the queen’s public key, /home/beehive/certs/sampleco.pem to the client. For
this example, we will assume the client will store the public key as /home/jbloggs/
certs/sampleco.pem.
2
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/sampleco.pem
Scenario 4: Client must present a valid, CA-signed certificate
This scenario presents a more strict regime that requires all clients to present CA-signed
certificates. In other words, clients cannot connect using a self-signed certificate, nor can they
connect using a copy of the queen’s public key.
Queen-Side Settings
Do the following:
82
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
1
Get the root certificate of the CA (certificate authority) that signed your client’s certificate.
Save the root certificate it on the queen. For this example, we will save it as /home/
beehive/certs/ca-cert.pem on the queen.
2
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/ca-cert.pem
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
There is no need to set the trustedCAPath parameter if you use a single root certificate
for all clients.
Variation: If your client’s certificates were not all signed by the same CA, then you must set
nCluster to recognize all the CA root certificates used to sign you clients’ certificates, like so:
1
Save the root certificates of all the signing CAs on the queen.
2
Set trustedCAPath to point to the directory that contains the root certificates. For
example:
•
3
trustedCAPath=/home/beehive/certs
Un-set the queen configuration parameter, trustedCAFileName, by setting it to no value
at all. For example:
•
trustedCAFileName=
Client-Side Settings
Do the following:
1
Save the client’s public key on the client. For this example, we will assume the client will
store its public key as /home/jbloggs/certs/my-client-cert.pem.
2
Make the following ODBC settings on the client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=0
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/my-client-cert.pem
Repeat the above steps for all ODBC client machines in your environment.
Encrypting Data Traffic
By default, Aster Database encrypts only control-path communications, such as the login
credentials an ODBC user submits, and the queries he or she submits. For implementations
that demand greater security, Aster Database also gives you the option of SSL-encrypting the
data traffic as the queen returns it to the client.
Aster Client Guide
83
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
All other things being equal, switching any network connection from unencrypted to SSLencrypted has the effect of reducing the maximum available rate of data transmission on that
connection.
To set this up:
Queen-Side Settings
Make the following settings on the queen:
•
secureWrites=true
Client-Side Settings
Make the following settings on each ODBC client:
•
SSLEncryptReads=1
Adding AD-Based SSO Authentication to SSL-Secured Aster Database
If your Aster Database is configured to authenticate users against Active Directory (“AD”), you
can configure your Aster Database ODBC clients to authenticate against AD, too. With this
configuration in place, each ODBC client will be required to authenticate against AD when it
tries to connect to Aster Database. If the ODBC client authenticates successfully, an SSL
channel is established automatically for communication between Aster Database and the
client. To set this up:
Queen-Side Settings
Make the following settings on the queen:
1
Set up Aster Database user authentication as explained in the Aster Database User Guide.
2
Configure the queen SSL configuration flags as described in one of the scenarios above
which you are planning on implementing. (For example, see “Scenario 1: Allowing
connections from clients without certificates” on page 80
Client-Side Settings
Do the following:
1
Set the flags in the client’s ODBC configuration file or registry as described in the scenario.
2
Set the EnableSSO flag in the client’s ODBC configuration file:
•
3
84
EnableSSO=1
If EnableSSO is set to 1, you must also:
•
Ensure that ServerIP is set to the fully qualified domain name of the Aster Database
queen and not to an IP address.
•
For 64-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so is
present at /opt/quest/lib64/. If /opt/quest/lib64/libvas-gssapi.so does
not exist, locate libvas-gssapi.so by referring to the VAS documentation and set
the GSSPath parameter to point to the installed location of libvas-gssapi.so. For
example, if libvas-gssapi.so is deployed at /usr/lib64, then the GSSPath
parameter needs to be set to /usr/lib64 in the ODBC.ini config file as shown below:
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
GSSPath=/usr/lib64
•
For 32-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so is
present at /opt/quest/lib/. If /opt/quest/lib/libvas-gssapi.so does not
exist, locate libvas-gssapi.so by referring to the VAS documentation and set the
GSSPath parameter to point to the installed location of libvas-gssapi.so. For
example, if libvas-gssapi.so is deployed at /usr/lib, then the GSSPath parameter
needs to be set to /usr/lib in the ODBC.ini config file as shown below:
GSSPath=/usr/lib
How to Set Configuration Parameters on the Queen
Many procedures in this document ask you to set configuration parameters on the Aster
Database queen. Below, we explain how to do this.
Persistent Setting of Parameters
To edit a setting (and have your edit survive reboots), make the setting as shown below.
Settings you change in this manner will survive reboots and soft restarts but they will not
survive upgrades.
1
Login as root into queen and use a text editor to open the file, /home/beehive/config/
procmgmtConfigs/coordinator.cfg
2
Find the “queenExec” section which looks like:
"taskName": "queenExec",
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1
3
Add the executableArgs section to the above section as shown below:
"executableArgs": "<flags in CSV format>"
For example, executableArgs for Scenario 1 (described earlier in this chapter) will look
like:
"taskName": "queenExec",
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1,
"executableArgs": "--disallowPeerWithoutCertificates=false,
--allowSelfSignedPeer=true,--trustedCAFileName=/home/beehive/certs/
server.pem,--sslCertificatePath=/home/beehive/certs/server.cert,
--sslPrivateKeyPath=/home/beehive/certs/server.key,--sslFileType=1"
4
Soft restart and activate the cluster.
How to Set Configuration Parameters on the Client
This section explains how to set up the 2nd-generation Aster Database ODBC client for SSL
communications with Aster Database. Consult the section that applies to your operating
system:
Aster Client Guide
•
“Linux ODBC DSN Set-Up”, below; or
•
“Windows ODBC DSN Set-Up” on page 86
85
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
Linux ODBC DSN Set-Up
On a UNIX-like systems, DSNs are added by setting parameters in the odbc.ini file
Sample ODBC.INI:
This sample assumes your queen machine is called cqueen.asterengqa.com and that you
are following Scenario 1 (outlined earlier in this chapter):
[ODBC Data Sources]
AsterTest=AsterDriverTest
[AsterTest]
Driver=AsterDriverTest
SERVER=cqueen.asterengqa.com
DATABASE=beehive
PORT=2406
UID=testuser13
PWD=testuser133
SQLSupportedConversions=3
NumericAndDecimalAsDouble=1
EnableSSO=0
GSSPath=
EnableSSL=1
SSLEncryptReads=0
SSLAllowSelfSignedPeer=1
SSLFileType=SSL_FILETYPE_PEM
SSLPrivateKeyPath=
SSLCertificatePath=
SSLTrustedCADir=
SSLTrustedCAFilename=
Sample ODBCINST.INI:
This sample assumes you have installed the driver in /Drivers/AsterDriver/ODBCDriver:
[AsterDriverTest]
Driver=/Drivers/AsterDriver/ODBCDriver/libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
Sample aster.ini:
This sample assumes you want to log error messages in /Drivers/AsterDriver:
[driver]
DriverManagerEncoding=UTF-32
DSILogging=1
ErrorMessagesPath=/Drivers/AsterDriver/ErrorMessages
Windows ODBC DSN Set-Up
On Windows, DSNs are added by modifying the Windows Registry using regedit.exe or
using a .reg file. The standards surrounding key names (such as whether to use “Server” or
“servername”) used in a connection string for ODBC driver are loose, so please take care to
follow the examples we provide.
Details for setting the values in the registry are given below. Choose the section that fits your
needs and client type:
86
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
•
“Adding a driver to a Windows 32-bit operating system”, below.
•
“Adding a 64-bit driver to a Windows 64-bit operating system” on page 88
•
“Adding a 32-bit driver to a Windows 64 bit operating system” on page 89
Adding a driver to a Windows 32-bit operating system
Make the following registry settings:
Driver for 32-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\AsterDriver32]
"Driver"="C:\\AsterDriver-Win32\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the
local machine and what the name of the driver should be. The values above are based on the
assumption that the driver folder is at "C:\AsterDriver-Win32" and the name of the driver is
"AsterDriver32". For an example .reg file that makes these settings, contact Teradata support.
DSN for 32-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
"AsterDSN32"="AsterDriver32"
Aster Client Guide
87
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
Adding a 64-bit driver to a Windows 64-bit operating system
Below we list the registry entries for setting up a 64-bit driver on Windows. Make the registry
settings shown below:
•
64-bit driver for 64-bit Windows
•
64-bit DSN for 64-bit Windows
64-bit driver for 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\AsterDriver64]
"Driver"="C:\\AsterDriver-Win64\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver64"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win64\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
64-bit DSN for 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN64]
"driver"="AsterDriver64"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
"AsterDSN64"="AsterDriver64"
Above, the name of the DSN for the 64-bit driver is AsterDSN64. The server being connected
to is 10.51.12.100.
88
Aster Client Guide
Connect Using Database Drivers
SSL Security for Aster Database-Client Communications
Adding a 32-bit driver to a Windows 64 bit operating system
Some applications running on a Windows 64-bit machine require a 32-bit driver. The 32-bit
operations work on Windows 64 bit machines under a mechanism called “Wow6432Node.”
Below we list the registry entries for setting up 32-bit drivers on Windows. Make the registry
settings shown below:
•
32-bit driver on 64-bit Windows
•
32-bit DSN on 64-bit Windows
32-bit driver on 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\AsterDriver32
]
"Driver"="C:\\AsterDriver-Win32\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the
local machine and what the name of the driver should be. The values above are based on the
assumption that the driver folder is at "C:\AsterDriver-Win32" for 32 bit drivers and the name
of the driver is "AsterDriver32". For an example .reg file that makes these settings, contact
Teradata support.
32-bit DSN on 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
Aster Client Guide
89
Connect Using Database Drivers
Process SQL Statements in JDBC
"SSLTrustedCAFilename"="\"\""
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\ODBC Data
Sources]
"AsterDSN32"="AsterDriver32"
Above, the name of the DSN for the 32-bit driver is AsterDSN32. The server being connected
to is 10.51.12.100.
Creating Certificates
Creating a Self-Signed Certificate
You may to create your own self-signed certificate. The high-level steps for doing this using
the OpenSSL tool are shown here, assuming you create a private key file called “host.key”:
openssl genrsa 1024 > host.key
chmod 400 host.key
openssl req \-new \-x509 \-nodes \-sha1 \-days 365 \-key host.key >
host.cert
Regenerating SSL Certificates/Private Key During Aster Database
Installation
You may manually regenerate the default private key and certificate using the following
command on the queen:
1
Working as root user, perform a soft shutdown on Aster Database:
# ncli system softshutdown
2
Wait for the system to shut down, and then run the resetCert command:
# /home/beehive/bin/lib/configure/ConfigureNCluster.py --resetCert
This will back up any existing server.key/server.cert files present in /home/beehive/certs
and generate a new private key and certificate file.
Process SQL Statements in JDBC
You can execute SQL statements by using a Statement or a PreparedStatement instance.
Sometimes it is more convenient to use a PreparedStatement object for sending SQL
statements to the database. This special type of statement is derived from the more general
class, Statement, that you already know.
If you want to execute a Statement object many times, it normally reduces execution time to
use a PreparedStatement object instead.
The main feature of a PreparedStatement object is that, unlike a Statement object, it is
given an SQL statement when it is created. The advantage to this is that in most cases, this SQL
statement is sent to the DBMS right away, where it is compiled. As a result, the
PreparedStatement object contains not just an SQL statement, but an SQL statement that
has been precompiled. This means that when the PreparedStatement is executed, the
90
Aster Client Guide
Connect Using Database Drivers
Process SQL Statements in JDBC
DBMS can just run the PreparedStatement SQL statement without having to compile it
first.
Process a Simple Query in JDBC
This example issueS a simple query and printS out the first column of each row using
a Statement.
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery("SELECT * FROM mytable WHERE columnf =
500");
while (rs.next()) {
System.out.print("Column 1 returned ");
System.out.println(rs.getString(1));
}
rs.close();
st.close();
This example issues the same query as before but uses a PreparedStatement and a bind
value in the query.
int fvalue = 500;
PreparedStatement st = conn.prepareStatement("SELECT * FROM mytable
WHERE columnf = ?");
st.setInt(1, fvalue);
ResultSet rs = st.executeQuery();
while (rs.next()) {
System.out.print("Column 1 returned ");
System.out.println(rs.getString(1));
}
rs.close();
st.close();
The instance returns a ResultSet containing the results. Aster Database does not support
cursor-based server-side caching of results which would save a small amount of time in
parsing.
To retrieve data from the ResultSet instance, call next(), which returns true if there are
results.
The ResultSet instance can be closed by calling close(), or if another query is issued using
the same Statement instance that was used to create this ResultSet.
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
You can also run Update/Insert/Delete statements and Create/Drop Table statements using the
same method.
Aster Client Guide
91
Connect Using Database Drivers
JDBC Troubleshooting and Limitations
JDBC Troubleshooting and Limitations
Infinity and Negative Infinity Timestamp Conversion
The JDBC driver converts 'infinity' and '-infinity' to specific TimeStamp values (a very large
and a very tiny TimeStamp) in the ResultSet.getObject API, which can cause unexpected
results on the client side.
Aqua Data Studio and other client tools, like SQL Assistant JAVA edition also have this issue.
These tools do not support 'infinity' and '-infinity' in the timestamp datatype. If they
encounter these values, they will display specific TimeStamp values (a very large and a very
tiny TimeStamp).
If you want to retain the negative infinity and infinity values, you can use ResultSet.getString
instead. Then you will be able to get the correct values for infinity and negative infinity,
without this conversion to a very small or very large timestamp value.
Thread Safety
The Aster legacy JDBC driver (pre-5.0.3) supports thread safety for all JDBC objects. The new
Aster JDBC Driver (5.0.3 and later), does not support JDBC object thread safety in all cases.
If you have developed code based on the legacy JDBC driver which uses the same instance of a
JDBC object across multiple threads simultaneously, you may have to modify your code to
ensure that JDBC object thread safety is implemented by your application.
Connect Reporting Tools to Aster Database
This section shows how to connect various reporting and query tools to Aster Database:
•
Connect Aqua Data Studio to Aster Database (page 92)
•
Connect MicroStrategy to Aster Database (page 93)
•
See also: Connect Using Database Drivers (page 43)
Connect Aqua Data Studio to Aster Database
ADS support is being deprecated.
AquaFold’s ADS lets you perform DDL operations and query data interactively, and it
provides tools that help you write and manage queries efficiently. ADS is a third-party tool
available for purchase from Aqua Fold directly.
Aster Database is compatible with ADS version 10.0.2 with patch ads-10.0.7_03-patch.zip.
92
Aster Client Guide
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
Install ADS
This section explains how to install ADS on your client workstation and connect to an Aster
Database.
1
Download Aqua Data Studio version 10.0.2 or later from http://www.aquafold.com/
downloads.html
2
Install Aqua Data Studio on your client workstation as explained in your version of the
Aqua Data Studio documentation at http://docs.aquafold.com/
aquadatastudio_11_documentation.html
3
Start Aqua Data Studio.
4
Select the command Server: Register Server.
5
In the list, select the “Aster nCluster Driver.”
6
Fill in the tabs as required.
Apply the ADS Patch
1
Download the patch from AquaFold:
http://dd1.aquafold.com/download/v10.0.0/ads-10.0.7_03-patch.zip
2
Unzip the patch files.
3
Replace the files under C:\Program Files\Aqua Data Studio 10.0 - 64bit\lib with the files
from ads-10.0.7_03-patch.zip.
4
Restart ADS.
Connect MicroStrategy to Aster Database
Observe the following guidelines when connecting MicroStrategy to Aster Database:
Versions
MicroStrategy 9 or later is required, and Aster Database 5.0 or later is required.
Platforms Supported
Aster Database supports Intelligence Server clients running on Windows XP and Windows
Vista with the Aster Database ODBC driver for Windows.
Limitations
Aster Database is only certified as a warehouse with MicroStrategy. Aster Database cannot be
used as a repository.
Set-up Instructions
To connect MicroStrategy to Aster Database, follow the steps below.
Prerequisites
Aster Client Guide
1
Microstrategy supports the ODBC Driver Manager bundled with the Aster Database
ODBC driver and the Windows Driver Manager only.
2
Make sure the following patches are applied to your MicroStrategy installation:
93
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
MicroStrategy 8: Contact MicroStrategy Customer support for
•
DATABASE.PDS file that's certified with Aster Database
•
DTMAPPING.PDS file that's certified with Aster Database
MicroStrategy 9: Contact MicroStrategy Customer support for
•
DTMAPPING.PDS file that works with Aster Database
MicroStrategy 9.0.1 and above: No changes required
Install Drivers
On the client machine where MicroStrategy runs, install the database drivers:
1
Install the Aster Database ODBC driver. See “ODBC Driver” on page 44 for installation
instructions.
2
Install the MicroStrategy VLDB driver, version 9 or later.
Connecting to an Aster Database
1
Open your MicroStrategy project and choose Intelligence Server as your connection option.
2
Use the Configuration Wizard to create a New Project Source.
3
Create a New Database Instance and give it a name.
4
In the Connection Type field, choose Aster Database nCluster.
5
Create a New Database Connection and give it a name.
6
Create a new ODBC System DSN and give it a name.
Once you have saved the System DSN, you can create reports in MicroStrategy that use the
data from your Aster Database. In your Logical Table definitions in MicroStrategy, you can
create queries that use special Aster Database tools such as nPath.
Best Practices
By following the guidelines below, you can avoid common errors in Aster DatabaseMicroStrategy integration.
Schema changes
•
If your schema contains NUMERIC(X,0) type columns, you should replace these with
INT or BIGINT type columns for a higher probability of success with existing
MicroStrategy reports.
•
For your small- to medium-sized tables that have no BIGINT or INT columns, you should
create the tables in Aster Database as replicated dimension tables. This takes more space in
the cluster, but works better with MicroStrategy.
Operational items
When pointing an existing MicroStrategy report from another database to Aster Database, the
warehouse schema should be “UPDATED” the first time when pointed to an Aster Database
Instance. This updates the metadata inside MicroStrategy for correct MicroStrategy columns.
This is particularly important when a schema is ported from some other database to Aster
Database.
94
Aster Client Guide
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
SQL query changes
•
Aster Client Guide
To use Aster Database’s nPath and other custom SQL-MapReduce functions:
•
Reports must be created as free-form SQL type reports in MicroStrategy.
•
Teradata Aster recommends that you embed the query in an MicroStrategy logical
table and build the reports on top of that.
•
If there are existing reports that need to run on Aster Database, some tweaking may be
needed in the VLDB Settings of MicroStrategy depending on the complexity of the
dynamically generated queries. For example, you may need to turn off TEMP table
creation, and so on. This is very rare, but editing VLDB settings can be useful in some
cases.
•
If the following error appears in queries: “Unable to find function to convert from <x> to
<y>”, and if x and y are datatypes like int, float, and so on, then you may need a
workaround to explicitly cast the column types. Contact Teradata Support for help on this.
95
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
96
Aster Client Guide
CHAPTER 4
Tools for .NET Environments
Teradata Aster provides a suite of tools that enable you to use Aster Database as a data source
in your .NET applications and reports. These tools include:
•
nClusterDNProvider—A managed .NET Data Provider
(nClusterDNProviderInstaller_ng_i386.msi, nClusterDNProviderInstaller_ng_x64.msi, a
database driver for ADO.NET) that allows Microsoft SQL Server Integration Services
(SSIS) and .NET client applications to connect to an Aster Database server.
•
Tutorials, including the sample program program.cs that show how to use the Aster
Database tools for .NET in a Microsoft reporting and BI environment.
This section explains how to install these tools and how to start writing applications and
reports that use databases in Aster Database.
•
Installing the Aster Database ADO.NET Driver (nClusterDNProvider) (page 97)
•
Performing SSIS Data Loading with nClusterDNProvider (page 99)
•
Sample Program for ADO.NET (page 113)
Note: The nCluster OleDB Driver (nClusterOleDbInstaller_i386.msi) is no longer supported.
Installing the Aster Database ADO.NET Driver
(nClusterDNProvider)
This section describes how to install nClusterDNProvider, the Windows Aster Database
database driver for ADO.NET. This driver is a managed .NET Data Provider for Aster
Database and allows .NET client applications (for example, Console, WinForms, ASP.NET,
and Web Services) to send and retrieve data from Aster Database.
Installation Prerequisites
Make sure the .NET 2.0 framework is installed on your workstation before installing
nClusterDNProvider. You can download it from: http://www.microsoft.com/downloads
Installing nClusterDNProvider
Follow the steps below to install nClusterDNProvider:
Aster Client Guide
97
Tools for .NET Environments
Installing the Aster Database ADO.NET Driver (nClusterDNProvider)
1
Get the Aster ADO.NET driver installer, whose name depends on your operating system:
•
For 32-bit, the installer is nClusterDNProviderInstaller_i386.msi
•
For 64-bit, the installer is nClusterDNProviderInstaller_x64.msi
by doing one of the following:
•
Copy the client package for your operating system from your queen node. Clients are
located in the directory /home/beehive/clients_all on the queen.
•
Download it from http://downloads.teradata.com/download/tools.
2
Place the installer on your client machine.
3
Run the installer.
a
In the Welcome screen, click Next.
b
In the Select Installation Folder screen:
Choose the installation location.
Specify who should be allowed to use the driver.
Click Next.
c
In the Confirm window, click Next.
d
When the Installation Complete window appears, click Close.
Note: Before proceeding, make sure that nClusterDNProvider is installed correctly. There should be several DLL files
and the RegisterAsterProvider.exe file in the .NET provider install path (for example, C:\Program Files (x86)\Aster
Data Systems\nCluster ADO.NET Provider (x86)). If not, nClusterDNProvider will not work. You can also check the
installation from the Control Panel. If the size of nClusterADO.Net Provider (x86) is less than 1 MB and the version is
1.0.x, you need a new driver.
Installing SQLServer and SSAS after installing the ADO.NET
provider
It is recommended to install SQLServer and SSAS before installing the Aster Database
ADO.NET provider. When you install the ADO.NET provider, the installer checks the
Windows registry to see if SSAS is installed. Then the correct files are installed and configured
automatically. But in cases where the ADO.NET provider is already installed before SSAS/SQL
Server, you will need to do the following steps before they can work together:
1
Find these four files:
•
Install.cmd
•
Install.vbs
•
Install.reg
•
nCluster.xsl
These files can be found in the folder where the Aster Database ADO.NET Provider is
installed, typically something like C:\Program Files (x86)\Aster Data Systems\nCluster
ADO.NET Provider (x86).
2
98
Install the files manually.
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
When installed, these files enable the Aster Database ADO.NET provider to appear as an
option within Microsoft Visual Studio and Microsoft Analysis Services.
To install the files, run the Install.cmd file from the command line with the /codebase
option. The /codebase option should have the full path to the Simba.Net.dll file. The
syntax to run the command is:
Install.cmd /codebase <full path>\Simba.Net.dll
For example, the command will look similar to:
Install.cmd /codebase "C:\Program Files (x86)\Aster Data
Systems\nCluster ADO.NET Provider (x86)\Simba.Net.dll"
If you are installing the ADO.NET provider to work with SQL Server 2005, you must use
the /vs2005 option, as in:
Install.cmd /vs2005 /codebase <full path>\Simba.Net.dll
3
Put the Aster Database cartridge (the file nCluster.xsl) into the corresponding directory for
SSAS. This file should be placed into all the cartridge directories for SSAS, which may
include all of the following directories, though the directories on your Windows machine
may differ slightly:
SQL Server 2008
C:\Program Files\Microsoft Analysis Services\AS OLEDB\10\Cartridges
C:\Program Files\Microsoft SQL Server\MSAS10.MSSQLSERVER\OLAP
\bin\Cartridges
C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\
IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
SQL Server 2005
C:\Program Files\Microsoft SQL Server\MSSQL.3\OLAP\bin\Cartridges
C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7
\IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 8\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
Sample Program to Demonstrate the Driver
The Program.cs program (“Sample Program for ADO.NET” on page 113) demonstrates the
features supported by nClusterDNProvider.
Performing SSIS Data Loading with
nClusterDNProvider
This section is a tutorial that shows you how to create a Data Flow task in SSIS for transferring
data from a table to a flat file via the Aster Database ADO.NET driver, nClusterDNProvider.
Overview
SSIS is a tool for building extract, transform, and load (ETL) jobs. In .NET environments,
SSIS provides a fast way to extract data from or load data into your Aster Database databases.
Aster Client Guide
99
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
To connect to Aster Database, SSIS requires the Aster Database driver, nClusterDNProvider.
This tutorial shows you how to set up SSIS to use the driver and provides an example for
exporting data from Aster Database to a flat file.
Procedure
To set up SSIS to use nClusterDNProvider, follow these steps:
1
Run Microsoft SSIS
2
Choose File: New > Project to create a new integration project.
3
In the New Project dialog:
4
a
Select Integration Services Project.
b
Enter your project a name
c
Click OK.
In the Connection Manager tab at the bottom of the window, add a new Connection.
To do this, right-click and select New ADO.NET Connection.
100
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
5
In the Configure ADO.NET Connection Manager window, click New.
In the next couple of steps we’ll create a widget (a database connection definition) that
allows SSIS to connect to your Aster Database.
Aster Client Guide
6
In the Connection Manager dialog, in the Provider drop-down box, choose Aster Data
Provider, and click OK.
7
In the Connection Manager dialog, make the following settings:
a
Set the Database to the name of the database in Aster Database you want to connect to.
b
Set the Host to your Aster Database queen IP address.
c
Set Port to the queen port, which is usually 2406.
d
Set the User Id and Password to the credentials of a user with sufficient rights on your
database in Aster Database.
e
Click OK. Note the name of the connection definition you are saving. Click OK again.
101
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
8
Choose View > Toolbox.
9
Drag and drop the Data Flow Task from the Control Flow Items list into the Control Flow
designer panel.
10 Click the Data Flow tab.
102
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
11 From the Data Flow Sources panel, drag an ADO NET Source object into the Data Flow panel.
12 To make this source a connection to your database in Aster Database,
double-click ADO NET Source to configure it.
The ADO.NET Source Editor dialog box appears.
Aster Client Guide
103
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
13 In the ADO.NET Source Editor dialog box, configure the Connection Manager properties:
a
Click Connection Managers.
b
From the Data access mode drop-down menu, choose SQL command.
c
In the SQL command text field, enter a SQL query.
14 In the ADO.NET Source Editor dialog box, check the Columns property values:
104
a
Click Columns.
b
Check the External Column and Output Column values.
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
15 In the ADO.NET Source Editor dialog box, configure the Error Output properties:
a
Click Error Output.
b
For each output column, set its Error property to Redirect row.
16 In the ADO.NET Source Editor dialog box, click OK.
Aster Client Guide
105
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
17 From the Data Flow Destinations panel, drag a Flat File Destination object into the Data Flow
panel.
18 Connect the green arrow of the ADO NET Source object to the Flat File Destination object.
106
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
19 Configure the Flat File Destination object.
Aster Client Guide
a
Double-click the Flat File Destination object.
b
In the Flat File Destination Editor, click Connection Manager.
c
Click New.
d
In the Flat File Format dialog box, click Delimited.
e
Click OK.
107
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
108
f
In the Flat File Connection Manager Editor dialog box, in the Connection manager name
field, enter the name of the output file of the connection manager.
g
Click Browse.
h
Select an output file and click Open.
i
Click OK.
j
In the Flat File Destination Editor, click Mappings.
k
Set the correct column mappings for the Flat File Destination object.
l
Click OK.
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
20 From the Data Flow Destinations panel, drag a Flat File Destination object into the Data Flow
panel.
This object is the destination for all error records.
21 Connect the error output (red arrow) of the ADO NET Source object to Flat File
Destination you just added.
22 Create another Flat File Connection Manager for the new Flat File Destination object that serves
as the error destination.
Aster Client Guide
a
Double-click the Flat File Destination object.
b
In the Flat File Destination Editor, click Connection Manager.
c
Click New.
d
In the Flat File Format dialog box, click Delimited.
e
Click OK.
f
In the Flat File Connection Manager Editor dialog box, in the Connection manager name
field, enter the name of the output file to be used to store all error records.
g
Click Browse.
h
Select an output file and click Open.
i
Click OK.
j
In the Flat File Destination Editor, click Mappings.
109
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
k
Set the correct column mappings for the error destination.
l
Click OK.
23 Save the project.
24 Run the project with debugging (Debug > Start Debugging).
You can check the progress of the workflow in the Progress panel.
110
Aster Client Guide
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
When the export is successful, the flat file source and destination objects in the Data Flow
pane turn green.
If the export is not successful, the ADO NET Source object turns red. For example, you
might get an exception like “Error: 0xC0047062 at Data Flow Task, ADO NET Source [16]:
System.ArgumentException: Error loading assembly: C:\Program Files (x86)\Aster Data
Systems\nCluster ADO.NET Provider (x86)\AsterDataC#DSII.dll.”
In this case, configure the 32-bit Runtime in SSIS for the project or you install the 64-bit
NClusterDNProvider.
The Microsoft SQL Server Business Intelligence Studio (BIDS) is a Visual Studio plug in.
For SQL Server 2008, this edition is 32-bit only. This requires using 32-bit drivers,
including for Aster Database.
However, in the 64-bit versions of Windows, there is a project flag that you must set to
allow 32-bit drivers to operate. Select the project, right-click, and choose Properties. Then,
set Run64BitRunTime to False. If not, you get an architecture mismatch and other
connection errors.
Aster Client Guide
111
Tools for .NET Environments
Performing SSIS Data Loading with nClusterDNProvider
Similarly, to run the package outside of BIDS—such as running a SQL Server Agent job or
running the package by itself—click the tab Execution options and check the check box Use
32 bit run time.
112
Aster Client Guide
Tools for .NET Environments
Sample Program for ADO.NET
Sample Program for ADO.NET
This section provides a sample program, Program.cs that demonstrates some of the features
supported by the Aster Database ADO.NET driver, nClusterDNProvider.
Features Demonstrated in the Sample
The Program.cs sample shows the following:
•
Insert into int column
•
DataReader
•
DataSet
•
Prepared query execution
•
Serializing to XML
•
CopyIn
•
CopyOut
•
Metadata operations
•
Indefinite timeout
•
Exception handling
Steps to Build and Test the Sample Program
1
Create a new Visual C# Console Application project called “Program.cs”.
2
Replace the generated Program.cs with the Program.cs provided below.
3
Ensure that the reference to nClusterDNProvider is properly set. If not, then you can do a
References: Add Reference and browse to nClusterDNProvider.dll.
4
In the main() method of Program.cs, edit the connectionString values to use your
Aster Database user credentials.
5
Compile and run the application.
Source Code of Program to Demonstrate nClusterDNProvider
Below is the source code for the sample program, “program.cs” that demonstrates the use of
the Aster Database ADO.NET driver.
using
using
using
using
using
using
using
System;
System.Collections.Generic;
System.Text;
System.Data;
System.Data.Common;
System.IO;
System.Threading;
namespace ConsoleApplication2
{
class Program
{
Aster Client Guide
113
Tools for .NET Environments
Sample Program for ADO.NET
// This method expects the following table in the backend:
//
// create table tableout(field_int int, field_numeric numeric)
// distribute by hash(field_int);
//
//
/*
static void AddWithDataSet(DbConnection connection)
{
DataSet ds = new DataSet();
nClusterDataAdapter da = new nClusterDataAdapter("select * from tableout", conn);
da.InsertCommand = new DbCommand("insert into tableout(field_int, field_numeric)
" + " values (:a, :b)", conn);
da.InsertCommand.Parameters.Add(new nClusterParameter("a",DbType.Int32));
da.InsertCommand.Parameters.Add(new nClusterParameter("b",DbType.Decimal));
da.InsertCommand.Parameters[0].Direction = ParameterDirection.Input;
da.InsertCommand.Parameters[1].Direction = ParameterDirection.Input;
da.InsertCommand.Parameters[0].SourceColumn = "field_int";
da.InsertCommand.Parameters[1].SourceColumn = "field_numeric";
da.Fill(ds);
DataTable dt = ds.Tables[0];
DataRow dr = dt.NewRow();
dr["field_int"] = 4;
dr["field_numeric"] = 7.3M;
dt.Rows.Add(dr);
DataSet ds2 = ds.GetChanges();
da.Update(ds2);
ds.Merge(ds2);
ds.AcceptChanges();
}
static void GenerateXmlFromDataSet(DbConnection connection)
{
nClusterDataAdapter da = new nClusterDataAdapter("select * from tableb",
connection);
DataSet ds = new DataSet();
da.Fill(ds);
ds.WriteXml("DataSetFeed.xml");
}
*/
// table schema: create table tableBytea (field_int int, field_bytea bytea)
distribute by hash(field_int);
static void ByteaDataType(DbConnection connection, String filename)
{
int res;
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(new BufferedStream(fs));
Byte[] bytes = br.ReadBytes((Int32)fs.Length);
Console.WriteLine(fs.Length);
br.Close();
fs.Close();
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "insert into tableBytea values(1, ?)";
DbParameter param = cmd.CreateParameter();
param.ParameterName = "bytesData";
param.DbType = DbType.Binary;
param.Value = bytes;
cmd.Parameters.Add(param);
res = cmd.ExecuteNonQuery();
114
Aster Client Guide
Tools for .NET Environments
Sample Program for ADO.NET
cmd.CommandText = "select field_bytea from tableBytea where field_int = 1;";
Byte[] result = (Byte[])cmd.ExecuteScalar();
fs = new FileStream(filename + ".suffix", FileMode.Create, FileAccess.Write);
BinaryWriter bw = new BinaryWriter(new BufferedStream(fs));
bw.Write(result);
bw.Flush();
fs.Close();
bw.Close();
}
private static void CopyIn(DbConnection connection)
{
Console.WriteLine("Creating command...");
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "COPY t_test_adonet_in from 'foo.out'";
Console.WriteLine("Command created: " + cmd.CommandText);
Console.WriteLine();
Console.WriteLine("ExecuteNonQuery: " + cmd.CommandText);
cmd.ExecuteNonQuery();
}
private static void CopyOut(DbConnection connection)
{
Console.WriteLine("Creating command...");
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "COPY t_test_adonet to 'foo.out'";
Console.WriteLine("Command created: " + cmd.CommandText);
Console.WriteLine();
Console.WriteLine("ExecuteNonQuery: " + cmd.CommandText);
cmd.ExecuteNonQuery();
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void SimpleInsert(DbConnection connection)
{
DbCommand command = connection.CreateCommand();
command.CommandText = "insert into t_test_adonet values(11, 'eleven')";
Int32 rowsaffected;
rowsaffected = command.ExecuteNonQuery();
Console.WriteLine("{0} rows added in table t_test_adonet", rowsaffected);
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void DataReader(DbConnection connection)
{
DbCommand command = connection.CreateCommand();
command.CommandText = "select * from t_test_adonet";
DbDataReader reader = command.ExecuteReader();
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
Console.Write("{0} \t", reader[i]);
}
Console.WriteLine();
}
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void PreparedQuery(DbConnection connection)
{
// Declare the parameter in the query string
DbCommand command = connection.CreateCommand();
Aster Client Guide
115
Tools for .NET Environments
Sample Program for ADO.NET
command.CommandText = "select * from t_test_adonet where i > ?";
// Now add the parameter to the parameter collection of
//the command specifying its type.
DbParameter param = command.CreateParameter();
param.ParameterName = "value1";
param.DbType = DbType.Int32;
command.Parameters.Add(param);
command.Parameters[0].Value = 2;
using (DbDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
Console.Write("{0} \t", reader[i]);
}
Console.WriteLine();
}
}
}
// table schema: create table t_test_adonet (i int, j char(10)) distribute by
hash(i);
static void OutputParameter(DbConnection connection)
{
Console.WriteLine("OuputParameter start....");
// Send a query to backend.
DbCommand command = connection.CreateCommand();
command.CommandText = "select * from t_test_adonet where i = 11";
// Now declare an output parameter to receive the first column of
// the tablea.
DbDataReader reader = command.ExecuteReader();
// Now, the firstcolumn parameter will have the value of
// the first column of the resultset.
Print(reader);
Console.WriteLine("OuputParameter end.");
}
static void Print(DataTable tbl)
{
// For each row, print the values of each column.
foreach (DataRow row in tbl.Rows)
{
for (int i = 0; i < tbl.Columns.Count; i++)
{
Console.Write("{0} \t", row[i]);
}
Console.WriteLine();
}
}
static void Print(DbDataReader reader)
{
// For each row, print the values of each column.
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write(String.Format("{0,12} |", reader.GetName(i)));
116
Aster Client Guide
Tools for .NET Environments
Sample Program for ADO.NET
}
// Write out the dividing line.
Console.WriteLine();
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write("==============");
}
Console.WriteLine();
// Write out the actual data.
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write("{0,12} |", (reader.IsDBNull(i)) ?
"<null>" : FormatData(reader[i]));
}
Console.WriteLine();
}
}
public static string FormatData(object obj)
{
if (obj is DBNull)
{
return "<null>";
}
if (obj is Byte[])
{
const string hexChars = "0123456789ABCDEF";
string binary
foreach (byte
{
binary +=
binary +=
}
= "0x";
b in (Byte[])obj)
hexChars[(b >> 4) & 0x0F];
hexChars[b & 0x0F];
return binary;
}
else
{
return obj.ToString();
}
}
static void GetMetaDataCollections(DbConnection connection)
{
DataTable tbl = connection.GetSchema("MetaDataCollections");
Print(tbl);
}
static void GetRestrictions(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Restrictions");
Print(tbl);
}
/*
static void GetDatabases(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Databases", new String[] {
"beehive" });
Aster Client Guide
117
Tools for .NET Environments
Sample Program for ADO.NET
Print(tbl);
}
*/
static void GetTables(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea";
Print(connection.GetSchema("Tables", restrictions));
}
static void GetColumns(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea";
Print(connection.GetSchema("Columns", restrictions));
}
static void GetViews(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea_view";
Print(connection.GetSchema("Tables", restrictions));
}
/*
static void GetUsers(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Users", new String[] { "beehive" });
Print(tbl);
}
* */
static void Setup(DbConnection connection)
{
Int32 rowsaffected;
Console.WriteLine("Creating the tables ...");
DbCommand command1 = connection.CreateCommand();
command1.CommandText = "DROP TABLE IF EXISTS tableBytea, tableout";
rowsaffected = command1.ExecuteNonQuery();
DbCommand command2 = connection.CreateCommand();
command2.CommandText = "CREATE TABLE t_test_adonet (i int, j char(10)) distribute
by hash(i)";
rowsaffected = command2.ExecuteNonQuery();
DbCommand command3 = connection.CreateCommand();
command3.CommandText = "CREATE TABLE tableBytea (field_int int, " + " field_bytea
bytea) " + "distribute by hash(field_int)";
rowsaffected = command3.ExecuteNonQuery();
DbCommand command4 = connection.CreateCommand();
command4.CommandText = "create table tableout(field_int int, " + "field_numeric
numeric) distribute by hash(field_int)";
rowsaffected = command4.ExecuteNonQuery();
}
static void Main(string[] args)
{
DbConnection connection = null;
//String connectionString = "Server=153.65.197.90;Port=2406;User
Id=beehive;Password=beehive;Database=beehive;CommandTimeout=-1";
//DbConnection conn = new nClusterConnection(connectionString);
try
118
Aster Client Guide
Tools for .NET Environments
Sample Program for ADO.NET
{
Console.WriteLine("Aster .Net Provider Test Program");
Console.WriteLine();
Console.WriteLine();
Console.WriteLine("Looking up provider factory...");
DbProviderFactory factory = DbProviderFactories.GetFactory("Aster.Net");
Console.WriteLine("Found provider factory.");
connection = factory.CreateConnection();
string connectionString =
"uid=beehive;pwd=beehive;dbname=beehive;ip=153.65.197.90;port=2406;NumericAndDecimalAsDo
uble=1";
connection.ConnectionString = connectionString;
// Connect to the database then retrieve the schema information.
Console.WriteLine("Connecting...");
connection.Open();
Console.WriteLine("Connected.");
Console.WriteLine();
// create the tables
Setup(connection);
// Metadata operations
GetMetaDataCollections(connection);
GetRestrictions(connection);
//GetDatabases(connection);
GetTables(connection);
GetColumns(connection);
GetViews(connection);
//GetUsers(connection);
DataReader(connection);
SimpleInsert(connection);
PreparedQuery(connection);
OutputParameter(connection);
//AddWithDataSet(connection);
// GenerateXmlFromDataSet(connection);
ByteaDataType(connection, "..\\..\\input\\out.txt");
CopyOut(connection);
CopyIn(connection);
}
catch (Exception e)
{
Console.WriteLine();
Console.WriteLine("***********************************************************");
Console.WriteLine("Exception: " + e.Message);
Console.WriteLine("Stack: ");
Console.WriteLine(e.StackTrace);
Console.WriteLine("***********************************************************");
if (connection != null)
connection.Close();
}
Console.WriteLine("Press any key to continue.");
Console.ReadKey();
}
}
}
Aster Client Guide
119
Tools for .NET Environments
Sample Program for ADO.NET
Possible Exceptions and Resolutions
These are resolutions to possible exceptions:
•
•
You might get an exception like “Exception: Unable to find the requested .Net Framework
Data Provider. It may not be installed.”
•
Reason: Your 32-bit provider is not installed or is broken.
•
Resolution: Reinstall your 32-bit provider and make sure that the size and version are
correct.
Loading assembly error. You might get an exception like:
Error -- 1056964601 : Internal error: The operation terminated
unsuccessfully.Error -- 1055784933 : Errors in the high-level
relational engine. The following exception occurred while the managed
IDbConnection interface was being used:
Error loading assembly:
C:\Program Files (x86)\Aster Data Systems\nCluster ADO.NET Provider
(x86)\AsterDataC#DSII.dll.
If this exception occurred in SSAS:
•
120
•
Reason: Your 64-bit provider might not be installed correctly.
•
Resolution: Reinstall your 64-bit provider and make sure that the size and version are
correct.
If this exception occurred in SSIS, then:
•
Reason: The 32-bit Runtime not enabled.
•
Resolution: Right-click the project and choose Properties. Then, set Run64BitRunTime to
False.
•
Note that in the SSAS view, no time zone information is shown for Aster Database
columns of the date data type “time with time zone”. This is a known issue.
Aster Client Guide
CHAPTER 5
Data Loading
To load data into Aster Database, you can use the Aster Database Loader Tool, the COPY
command, the INSERT command, or a custom-defined SQL-MapReduce data loading
function you have written. This section provides tips for efficient loading and shows how to
load using the Aster Database Loader Tool. The following sections explain these utilities:
•
Best Practices for Data Loading (page 121)
•
Aster Database Loader Tool (page 125)
•
Troubleshoot Loading (page 143)
Best Practices for Data Loading
This section describes best practices for efficiently loading data into Aster Database. In
particular, this section explains how to help optimize new loading projects using the tools
provided by the Aster Database loader nodes and the Aster Database Loader Tool
(ncluster_loader). The sections are:
•
Loading Terminology (page 121)
•
Scenario 1: Pre-Production Data Loading with Aster Database (page 122)
•
Scenario 2: Loading in a Production Environment with Aster Database (page 123)
Loading Terminology
We use the following terms in the text that follows:
Aster Database loader node: In the cluster, a loader node is a node dedicated to data loading.
Many loader nodes can operate in parallel.
Aster Database Loader Tool (ncluster_loader) is the client application for initiating highspeed bulk loads.
Aster Database Load Error Logging is a feature in ncluster_loader that allows you to perform
loading that is more tolerant of poorly formatted input data. Load Error Logging sends
malformed rows to an error logging table.
Input data: Source input file(s) which are to be loaded into Aster Database. All source files are
compatible with a format that Aster Database is able to load. Examples include the CSV
Aster Client Guide
121
Data Loading
Best Practices for Data Loading
format (RFC 4180: http://www.rfc-editor.org/rfc/rfc4180.txt) or the tab-delimited format. Each
file must use a consistent newline character throughout. Never mix UNIX and Windows-style
newline characters in the same file; this will cause your load attempt to fail!
Data staging node(s): Nodes where all the input data is present. In a typical setup the input
data is stored on the local filesystem. However, other use cases where all the data is stored on a
network-mounted storage device are possible.
Row(s): In any given input data file individual rows are present. The used format for the input
data describes where row boundaries are. Usually a “row” refers to a logical unit of
information that needs to be stored as a unit in the data warehouse. One example for rows
include call records which consist of source and destination phone numbers, call duration
time, and so on.
The following section highlights three different data loading scenarios and the respective best
practices that should be performed to make the process as a whole as seamless as possible.
Scenario 1: Pre-Production Data Loading with Aster Database
To validate your loading process and data before deploying it to your production system, it is
an accepted industry best practice to load to a pre-production or testing system. You can
perform test data loading using the standard Aster Database command-line, bulk loading tool,
ncluster_loader. For a complete overview of the tool’s command-line options refer to
“Argument Flags” on page 127.
The tool’s error logging features are useful for debugging your loading process. If there is a
possibility that your input data contains malformed rows, you should consider using error
logging for bulk load operations to accomplish the following goals. Below, we also show the
command-line options for each task:
1
Make sure that even in the presence of malformed rows a given load operation succeeds;
This can be accomplished by enabling error logging but not setting an error logging limit.
(Set --el-enabled but do not set an --el-limit.) If you are not interested in what
errors are present, malformed input rows can be discarded so that they are not stored in
the cluster (--el-enabled --el-discard-errors).
Tip: A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to abort. So if the
table you are loading into contains these constraints and your load is failing, check the data you are loading to ensure
it complies.
2
Store malformed rows into a separate table for later inspection;
You should specify a custom error logging table to store malformed rows. This is done via
the appropriate option (--el-enabled --el-table = 'my_errorlogging_table').
If the error logging feature of Aster Database is turned on without using --el-table to
specify an error logging table, malformed rows are stored in the default system error
logging tables (nc_errortable_part or nc_errortable_replicated). This is
undesirable because the rows from different loads will be mixed together in the default
122
Aster Client Guide
Data Loading
Best Practices for Data Loading
tables. Note that any custom error logging table has to inherit from the default system
error logging table. To create such a table, see .
3
Abort data load operation in the presence of too many malformed rows;
This is in particularly useful if you want a given load operation to abort if too many
malformed rows are present in the input data (--el-enabled --el-limit = 100). In
order to preserve atomicity for bulk load operations, the load operation fails as a
transaction when the error limit is reached. When the operation fails, any rows already
written by the transaction to the target table and error logging table are deleted.
4
Label malformed rows in the error logging table.
If multiple operations are loading data into Aster Database at the same time (e.g., the preproduction system is testing integration of two separate data sources), you can label each
load operation to identify which rows belong to which data source (--el-enabled -el-label = 'my_data_source').
The following types of errors in input data files can be detected:
1
Malformed datatypes (e.g., text value for an integer column);
2
Missing column values (e.g., the input data file provides data values for the first two
columns, but not for the third one);
3
Additional column values (e.g., the input data file contains a row with more values than
there are columns in the destination table);
4
Check constraint violations (e.g., the integer value of an imported data field exceeds the
range allowed by the check constraint on the target table);
5
Character set conversion errors (e.g., input data file contains invalid UTF-8 characters);
6
Missing or invalid distribution key (e.g., for your distribution key, you have specified a
column that has a distribution-key-disallowed datatype).
Summary
In an environment where not all operations are run in an automated fashion, where new data
sources are to be integrated, or where existing ETL processes are to be changed, we
recommend that you set an error logging limit to prevent accumulation of too many
malformed rows in the error logging tables. If malformed data rows need to be inspected, we
recommend that you use a custom error logging table or create a separate error logging table
for that load job.
Scenario 2: Loading in a Production Environment with Aster Database
Teradata Aster recommends that, before you bulk load data into your production system, you
first verify your loading procedure on a test system. In this section, we’ll focus on the
differences between a data loading job in a testing environment and one in a production
system.
In a production deployment, we recommend that you follow the steps below to ensure you
take advantage of the bulk loading and error logging tools in Aster Database:
Aster Client Guide
123
Data Loading
Best Practices for Data Loading
1
Performance when loading autopartitioned tables
Beginning in Aster Database version 5.0, loading of logically partitioned tables has much
lower overhead and performs better than previously. Specifically, performance is improved
when loading data into a table with many child partitions. The improvement applies to
tables that use either row or columnar storage, but is most pronounced for tables with row
storage.
2
Run ANALYZE after you load each reasonably sized batch.
When bulk loading data, it’s important to run ANALYZE regularly. In ncluster_loader, use
the --auto-analyze (or -z) flag to do this. With this flag present, ncluster_loader will
run ANALYZE on the target table or tables after it loads the data.
While running, ANALYZE requires an amount of disk space on the cluster proportional to
the amount of new data in the table being analyzed. For this reason, Teradata Aster
recommends that your run ANALYZE after every load, rather than waiting for multiple
loads to finish. Waiting too long can result in a large amount of the cluster’s disk space
being consumed during the running of ANALYZE. Daily child tables offer a good example:
When you load to daily child tables, run ANALYZE on each child table after you load the
day’s data into that table.
3
Stay informed about cluster activity.
As with any other production system component, it is important to stay informed about
any irregular activity that is going on in the cluster, in this case in particular what is
happening in the data loading path.
If you want to make sure that large loads with too many malformed rows do not complete
successfully, you can set an error logging limit for your load operation (--el-enabled -el-limit = '100000'). You can specify a limit clause with a value that is small enough
to allow most loads to complete successfully while forcing the failure of loads with
unexpected percentages of malformed rows. Exact values will depend on the size of the
files being loaded and the expected malformed row ratios.
You should use an alert system to monitor the size of your error logging tables. See for
information.
4
Keep an eye on the volume of data in the load error logging tables.
The system does not perform automatic truncation of the error logging tables! The error
logging tables are regular tables in Aster Database. The same operations that an
administrator would apply to regular tables thus need to be applied to error logging tables.
Tasks include the appropriate setup of child table hierarchies for efficient data processing,
regular VACUUM operations to free up disk space occupied by dead rows, and the
monitoring of table growth over time. Custom error logging tables can be used by passing
the appropriate arguments when you run the Aster Database Loader Tool (for example, -el-enabled --el-table = 'el_table_April_2010').
5
Use an error logging file for faster re-try of rows that failed.
Have ncluster_loader create a file containing the row data that failed to load. To do this,
pass the argument, --el-errfile <my_error_file.txt> to the tool. Upon
completion of the load, you can inspect the contents of the error logging file, fix issues you
find there, and reload from that file.
124
Aster Client Guide
Data Loading
Aster Database Loader Tool
6
Watch your load error statistics.
To monitor error rates, check the statistics tables, nc_all_errorlogging_stats and
nc_user_errorlogging_stats, as shown in . These tables show you the malformed and
well-formed row count for each load operation. Records from the statistics tables can be
correlated with the malformed rows in the load error logging tables. Both the error logging
and statistics tables contain an error logging label which can be used to identify a
particular loading operation.
7
Trace malformed input data to the data source.
An important aspect of a production system is that it must provide the user with the
means to find the problem source when an operation fails. When loading data, the user
can use the error information about malformed rows (SQL error code and error message,
sqlerrcode and errmessage, respectively, in the load error logging table) to diagnose
the problem in the input data. To do this, you must invoke your data loading operation
with the error logging feature enabled (--el-enabled). As described above, this
information can then be correlated with the error logging statistics table to find the source
of the malformed data. To facilitate this process, you may wish to apply custom error
logging labels to each load operation (--el-enabled --el-label =
'my_data_source').
Loading Best Practices Summary
For any production system, the close monitoring of system components is absolutely mission
critical. Correlating error information present in the error logging tables can reveal possible
problems early in the process.
Aster Database Loader Tool
The Aster Database Loader Tool, ncluster_loader, is a command-line application for bulkloading data from files into Aster Database. It provides an alternative to the SQL COPY
statement. Each invocation of the Aster Database Loader Tool is equivalent to a single
invocation of the COPY statement. Each invocation is a single database transaction that loads
the contents of the specified data file or files into the specified table or tables.
This section includes:
Aster Client Guide
•
Syntax (page 126)
•
Argument Flags (page 127)
•
Exit Status (page 132)
•
Install the Loader Tool (page 133)
•
Load Data with the Loader Tool (page 133)
•
Load from Multiple Files Using a Map File (page 135)
•
Examples (page 138)
•
Hints for Successful Loading (page 139)
•
Error Logging (page 142)
125
Data Loading
Aster Database Loader Tool
Syntax
To run the Aster Database Loader Tool, you type:
$ ncluster_loader [arguments] [schemaname.]tablename [ filename |
dirname ]
where
•
arguments are the command-line flags that control how the loader runs. The flags are
explained in Argument Flags, below, or you can display the help by typing:
$ ncluster_loader -?
•
schemaname is the optional name of the destination schema. If no schema name is
provided, Aster Database will search the schemas listed in the schema search path.
•
tablename is the name of the destination table (See “Case-Sensitive Handling for Table
Names” on page 126 if you wish to have Aster Database evaluate table names in a casesensitive manner.);
•
filename or dirname indicates the file or directory of files to be loaded.
•
Qualified path of the file containing the data to be loaded. The contents of
the file must be in either CSV or text format, as described for the COPY statement.
Details of the encoding used (such as non-default values for null or delimiter) are
specified using the appropriate options, as described below; or
•
dirname Qualified path of the directory containing one or more data files to be
loaded. All data files found within this directory are expected to be in the same format
and will be loaded as a single transaction. Subdirectories will not be processed.
filename
If you don’t supply a file or directory name argument, Aster Database Loader assumes you
want to load from STDIN. See “Load from STDIN Example” on page 138.
Tip for Windows users: When using Aster Database Loader on Microsoft Windows, bear in mind:
• If the filename or dirname contains spaces, make sure you enclose it in double quotes.
• When specifying a dirname, you must use a double backslash at the end of the path. For example, use
“c:\temp\loadFiles\\” to specify a directory called “loadFiles”.
• Never mix UNIX and Windows-style newline characters in the same data file. Doing so will cause your load attempt
to fail.
Case-Sensitive Handling for Table Names
To force Aster Database Loader to treat your table and schema names in a case-sensitive
manner, you must surround the schema and table names in double quote marks when you
invoke ncluster_loader. To pass a double quote mark to Aster Database Loader, you must
prefix it with the escape character, which is a backslash. That means you type a double quote
mark like so: \". If either the schema name, the table name or both names include capital
letters, you must surround each name in escaped quotation marks, individually.
For example, assuming you want to load to table "Bar" in schema "Foo", then you would
invoke Aster Database Loader using this form:
ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"Bar\"
mydata.csv -c
126
Aster Client Guide
Data Loading
Aster Database Loader Tool
and in the map file, you would reference the table as:
"table" : "\"Foo\".\"Bar\"",
If you want to load to table "bar" in schema "Foo", you would still need to escape quote the
schema and the table separately as follows:
ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"bar\"
mydata.csv -c
and in the map file, as:
"table" : "\"Foo\".\"bar\"",
Argument Flags
In addition to the schemaname.tablename and filename/dirname arguments explained
above, the Aster Database Loader Tool takes the following argument flags at the command line
or in the map file. (Map files let you load from many input files in a single running of Aster
Database Loader. See “Troubleshoot Loading” on page 143.)
In the table that follows, the argument flags are sorted based on the long-form, command-line
flag:
•
The left column lists the flag you use at the command line.
•
The middle column lists the flag you can use in a map file (See “Rules for Passing
Arguments in a Map File” on page 136.). If no value appears in the middle column, then
the argument is one that can only be passed at the command line.
Table 5 - 6: Argument Flags for Aster Database Loader Tool
Flag at Command Line
Flag in Map File
Description
-a [ --auto-partition]
auto-partition
Specifies that ncluster_loader will run in autopatitioning mode,
which automatically routes each row to the appropriate child table in
a parent/child table created through inheritance.
-B [ --begin-script ]
arg
begin-script
Specifies the qualified path of the file containing SQL commands
that should be executed when the transaction starts, i.e. immediately
after the BEGIN command is issued to Aster Database. Note: Datareturning statements such as SELECT are not allowed in scripts
executed by the ncluster_loader.
-c [ --csv ]
Aster Client Guide
Pass this argument if you want ncluster_loader to interpret the input
file as being in CSV format. By default ncluster_loader uses a text file
with a tab-delimited format, often called “TSV”.
127
Data Loading
Aster Database Loader Tool
Table 5 - 6: Argument Flags for Aster Database Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
-C [ --columns ] arg
columns
An optional comma-separated list of the columns in the target table
that will receive the data being loaded (the default is to assume that
each input row contains exactly one data value for each column in
the target table, in the order in which the table columns were
defined).
By using -C, you can specify that the ordering of the columns in the
input file is different from the ordering in the table. You can also use
-C to specify that this input file contains values for only some of the
columns in the table.
The input data is assumed to contain values for the columns in the
order specified here. For example, to load data into columns 'col1'
and 'col2', one could specify "col1, col2" as the value for this option.
Column names not specified here are expected to get NULL values.
See “Using the -C option with uppercase or special characters” on
page 144.
-d [ --dbname ] arg
dbname
The column delimiter character to use when interpreting the input
file (must be a string that represents a valid single character, such as
'd' or '\n'). The default is a tab character ('\t') in text mode, a
comma in CSV mode.
-D [ --delimiter ] arg
--date-style arg
The name of the database to connect to (default is "beehive").
date-style
Specifies the date format. If you are using a map file, you must
specify a single date-style per map file.
The following formats are supported:
• ISO - This is the default. Uses ISO 8601-style dates and times
(YYYY-MM-DD HH:MM:SS) like, for example,
2010-12-17 07:37:16-08
• SQL - Uses Oracle/Ingres-style dates and times. For example:
12/17/2010 07:37:16.00 PST
• POSTGRES - Uses traditional PostgreSQL format. For example:
Wed Dec 17 07:37:16 2010 PST
• German - Uses dd.mm.yyyy for numeric date representations. For
example:
17.12.2010 07:37:16.00 PST
• SQL,DMY - For example:
17/12/2010 15:37:16.00 CET
• SQL,MDY - For example:
12/17/2010 07:37:16.00 PST
• Postgres,DMY - For example:
Wed 17 Dec 07:37:16 2010 PST
-e [ --escape ] arg
Specifies the escape character for CSV input (must be a string that
represents a valid single character, such as d or \n). The default is the
quote value - double-quote by default. You may only pass this option
when loading a csv-formatted file (--csv).
Warning: ncluster_loader will fail if you pass this argument when
loading a tab-delimited text file.
128
Aster Client Guide
Data Loading
Aster Database Loader Tool
Table 5 - 6: Argument Flags for Aster Database Loader Tool (continued)
Flag at Command Line
Flag in Map File
-E [ --end-script ] arg end-script
Description
Qualified path of the file containing SQL commands that should be
executed when the transaction is about to commit, i.e., immediately
before the END/COMMIT command is issued to Aster Database.
Note: Data-returning statements such as SELECT are not allowed in
scripts executed by the Aster Database Loader Tool.
--el-discard-errors
discard-errors
or skiperrors
Sub-flag that can accompany --el-enabled. If present, specifies
that all errors (malformed rows) be discarded by Aster Database. By
default, errors are not discarded.
--el-enabled
errorlogging
If present, turns on error logging for this invocation of the
ncluster_loader. This needs to be enabled for any other error logging
option to be accepted. The default is disabled. See “Error Logging”
on page 142.
--el-label arg
label
Sub-flag that can accompany --el-enabled. The --el-label
flag specifies the label to be used to tag the malformed rows. By
default, Aster Database will use a statement identifier as the label
value. (There’s one statement identifier per COPY command; if there
is one map entry for many input files, then you’ll have a unique
statement identifier per input file.) The label is useful for finding
your failed rows in the error logging table and for finding statistics
about the load attempt in the nc_all_errorlogging_stats
table.
-F --el-errfile arg
errfile
Sub-flag that can accompany --el-enabled. The --elerrfile flag introduces the pathname of the optional error logging
file. If you use error logging, you must have an error logging table,
and you can have an error logging file. Upon completion of the load,
ncluster_loader writes the contents of the error logging table’s
rawdata column (and no other columns) to the error logging file.
Only the contents of this column are written to the file. The filename
will have a numeral appended to it in the form, “_0”.
For this option to work, you must have also specified an el-table.
Regardless of whether or not you specify that an error logging file
should be used, the error logging table will still contain all error rows
upon completion of each load.
-L [ --el-limit ] arg
limit
Sub-flag that can accompany --el-enabled. The --el-limit
flag sets the maximum number of malformed rows allowed for each
file being loaded. By default the limit is unbounded. If the error
count during your load attempt exceeds the specified --el-limit,
the transaction aborts and no rows are inserted into the destination
table, and no rows are inserted into the error logging table.
-S [ --el-schema ] arg
schema
Sub-flag that can accompany --el-enabled. The --el-schema
flag specifies the name of the schema to which the error logging table
belongs. If not specified, the public schema is used.
Aster Client Guide
129
Data Loading
Aster Database Loader Tool
Table 5 - 6: Argument Flags for Aster Database Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
-T [ --el-table ] arg
table
Sub-flag that can accompany --el-enabled. The --el-table
flag specifies the name of the error logging table that will capture
malformed rows.
If you are loading to a distributed table, the error logging table must
be a distributed table. Likewise, if you are loading to a replicated
table, the error logging table must be a replicated table.
If this flag is not provided, Aster Database uses a default table:
nc_errortable_repl is used if the load-destination table is a
replicated table, or nc_errortable_part is used if the loaddestination table is a distributed table.
Important: The system does not perform automatic truncation of the
error logging tables! You must VACUUM them regularly.
-f [ --force-loader ]
force-loader
filename (Not a flag; you pass file
the file or directory name itself!)
Instructs the Aster Database Loader Tool to use the loader node
specified with the -l or --loader option even if the IP address
provided is not known to Aster Database. Note that if this option is
specified, the Aster Database Loader Tool will only try that single IP
address and return an error status if the connection fails for any
reason.
Qualified pathname of the file or directory containing data to be
loaded. The contents of the file must be in either CSV or text format,
as described for the COPY statement.
If you pass a directory name, then all data files found in the directory
are loaded in a single transaction. Subdirectories are ignored. All files
must be in the same format.
If you pass no file or directory name, Aster Database Loader loads
from STDIN.
In a map file, use the file flag. At the command line, there is no flag
for this argument; instead, ncluster_loader assumes that you pass the
named-flag arguments first, then the table name, and finally the
source file name.
-? [ --help ]
Shows online help for Aster Database Loader Tool.
-h [ --hostname ] arg
Host name or IP address of the Aster Database queen. Default is
'localhost'. This always points to the queen, even if you’re loading
using loader nodes. See “Multiple Loader Nodes” on page 140.
--header-included
Indicates that the first line of the data file to be input is a comma
delimited list of the column names. The number of skip-rows will
be counted beginning after the header, if this option is specified.
Cannot be combined with --columns.
-l [ --loader ] arg
130
loader
Preferred loader IP address. If a value is provided, the Aster Database
Loader Tool tries to load to this IP address before trying any other
loader node. Note that the Aster Database Loader Tool expects this
IP address to be known by Aster Database and will ignore the address
provided if this is not the case. To change this behavior one can use
the -f or --force-loader flag.
Aster Client Guide
Data Loading
Aster Database Loader Tool
Table 5 - 6: Argument Flags for Aster Database Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
-m [ --map-file ] arg
Name of the file containing mappings of data files or directories to
target tables. This option allows you to load multiple directories into
a table and also to load data into multiple tables within the same
transaction. For details about the format used for the map file, see
“Troubleshoot Loading” on page 143.
-n [ --null ] arg
String that is used to represent a null value. The default is \N in text
(non-CSV) mode, and an empty value with no quotes in CSV mode.
When loading any non-CSV delimited format (e.g. TSV), you can
easily load files that contain empty strings (that is, files that don’t use
the typical \N to represent nulls). To do this, run
ncluster_loader with the -n flag, followed by two doublequote characters. That is, you type:
ncluster_loader -n ""
See also --null-backslash-escapes.
Indicates that backslashes in the '-n/--null' flag should be
treated as special characters, which will be processed according to the
default rules for escaped strings.
--null-backslashescapes
You should use this flag whenever the null string contains a
backslash, to ensure compatibility with future versions of Aster
Database.
TCP port on which Aster Database listens for connections. Default is
2406.
-p [ --port ] arg
-P [ --data-prefix ]
arg
data-prefix
String that should be used as a prefix for each row (each line in a data
file) being loaded into Aster Database.
Important: The Aster Database Loader Tool will simply concatenate
this value to each row without any parsing. You must insert the
proper delimiter, quote, and escape characters in the prefix string.
Character to be used as the quoting character for CSV input (must
be a string that represents a valid single character, such as d or \n).
The default is the double-quote. This option is only valid when CSV
input has been specified (-c or --csv) and will cause the load to
fail otherwise.
-q [ --quote ] arg
Note that if using a single quote (') as the quoting character, you
should escape it when specifying it with -q as -q"\'". Otherwise
any input data that includes a comma will cause the load to fail.
Specifies how many rows of the file to skip before starting to load
data. If combined with --header-included, --skip-rows will
not start counting until after the first line in the file. The default is 0.
--skip-rows arg
-t [ --timeout ] arg
timeout
Timeout value in seconds for Aster Database connection attempts.
Default is 30 seconds.
tablename (Not a flag; you
pass the table name itself!)
table
Name of the destination table. In a map file, use the table flag. At
the command line, there is no flag; instead, ncluster_loader assumes
that you pass the named-flag arguments first, then the table name,
and finally the source file name or directory name.
Aster Client Guide
131
Data Loading
Aster Database Loader Tool
Table 5 - 6: Argument Flags for Aster Database Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
--truncate-table
truncate-table
Performs a TRUNCATE operation to empty the table before
beginning the load. If child tables are present, they will be truncated,
too. By default, this feature is disabled.
-U [ --username ] arg
username
If present, the Aster Database Loader Tool connects to the database
with this username instead of the default ("beehive"). (The user
account must have permission to do so, of course.)
-v [ --vacuum-table ]
Runs VACUUM on the target table if the load fails. If target child
tables or partitions are present, they are VACUUMed, too. By
default, this feature is disabled.
-V [ --version ]
Print the Aster Database Loader Tool version and exit.
--verbose
Run in verbose mode.
-w [ --password ] arg
password
Connect to the database using the specified password (as opposed to
providing a password at the prompt).
-W [--password-prompt]
Forces the Aster Database Loader Tool to prompt for a password
before connecting to a database. It should automatically prompt for
a password whenever the server requests password authentication. If
no password prompt is issued and the server requires password
authentication, the connection attempt will fail.
-z [ --auto-analyze ]
Runs ANALYZE on the table or tables after loading the data. By
default, this is disabled. If data was loaded to child tables via
autopartitioning, they will be analyzed, as well.
Warning: Analyzing a columnar table may be slow if there are many
columns. To improve the speed of statistics collection, execute a
separate ANALYZE command after the load that only processes the
columns involved in query row filters or grouping.
Exit Status
Table 5 - 7: The Aster Database Loader Tool exit values
132
Exit
Value
Description
0
The Aster Database Loader Tool terminated successfully.
2
An error internal to the Aster Database Loader Tool was detected.
3
Another error was detected.
4
Failed to establish a connection to Aster Database.
5
An error was detected while communicating with Aster Database.
6
Malformed input data detected.
Aster Client Guide
Data Loading
Aster Database Loader Tool
Install the Loader Tool
Install the Aster Database Loader Tool on a data staging machine that has fast network
connectivity to your Aster Database loader nodes. The data staging machine is where you will
place the data files to be loaded. For a list of supported operating systems for the Aster
Database drivers and utilities, see the Aster Database Drivers and Utilities Guide. Note that if
you are upgrading, you can simply replace ncluster_loader with the newer version.
Obtain the Loader Tool Installer Package
To obtain the Aster Database Loader Tool package for your client operating system, download
it from one of these places onto your client machine in the directory where you will install it:
•
To get the newest package, download it from http://downloads.teradata.com/download/tools
•
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
Install the Loader Tool on Linux
1
Obtain the Loader Tool Installer Package.
2
Unzip the file and expand the archive.
3
Set permissions to make the ncluster_loader file executable.
4
On Linux, ncluster_loader requires glibc 2.7 or later. Update glibc if needed.
Install the Loader Tool on Windows
1
Obtain the Loader Tool Installer Package.
2
Unzip the file and expand the archive.
3
Run ncluster_loader.exe from the command line.
Install the Loader Tool on AIX
1
Obtain the Loader Tool Installer Package.
2
Unzip the file and expand the archive.
3
Set permissions to make the ncluster_loader file executable.
See also “AIX Client Dependent Libaries” on page 44.
Load Data with the Loader Tool
Connecting
The Aster Database Loader Tool is a regular Aster Database client application. In order to
connect to an Aster Database, you need to specify the host name or IP address of the queen
node via the command line option -h. If the connection could not be made for any reason
(e.g., insufficient privileges, server is not running on the targeted host, etc.), the Aster
Database Loader Tool will return an error and terminate.
Procedure:
To load data with the Aster Database Loader Tool, do this:
Aster Client Guide
133
Data Loading
Aster Database Loader Tool
1
Install the Aster Database Loader Tool on your data staging machine. (See “Install the
Loader Tool” on page 133.)
2
If you have not already created one or more loader nodes in Aster Database, create them
now.
3
Prepare the file or files that contain the data you wish to load:
a
For hints on file formatting, see the descriptions of the --csv, --delimiter, and -null arguments, below. Use a consistent newline character in your input file(s)!
b
Determine your mapping of the input file’s field values to the columns of your target
table. See the description of the --columns argument, below.
c
Determine any special parsing hints you need to provide to the loader routing. See the
descriptions of the --escape, --quote, and --data-prefix arguments, below.
4
Place your data input file(s) on the data staging machine.
5
Figure out how you want to handle records that fail to load. See the section “Error
Logging” on page 142. Teradata Aster recommends that you create an error logging table
that will receive rows that fail to load. Be prepared to query the
nc_all_errorlogging_stats table for statistics about your load attempt.
6
Figure out which advanced options of the Aster Database Loader Tool you will use:
7
a
Do you need to load from multiple files or insert into multiple tables? If so, see the
description of the --map-file argument and read the section, “Troubleshoot
Loading” on page 143.
b
Do you need to automatically partition data when loading parent child tables with
inheritance? Use the --auto-partition argument and read the section, “Loading
Parent Child Tables with Inheritance” on page 140
c
If you need to run an SQL script before and/or after the data is loaded, read the
descriptions of the --begin-script and --end-script arguments, below.
Load your data by running the Aster Database Loader Tool on your input file(s). See
“Argument Flags” on page 127 for a list of the command-line options. The command you
type will be similar to:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" customers
input_data.txt
8
Check the results of your load attempt by querying the statistics tables,
nc_all_errorlogging_stats and nc_user_errorlogging_stats.
9
Check your error logging table(s) for rows that failed to load. Be aware that if the load
failed entirely (for example, if the number of errors exceeded the --el-limit), then no
new rows will appear in your target table, and no error rows will appear in the error
logging table.
10 If you wish to import the failed rows, find each row of input data that failed, edit it to fix it,
and combine these fixed rows into a new input file. Re-run the Aster Database Loader Tool
on the new input file.
134
Aster Client Guide
Data Loading
Aster Database Loader Tool
Load from Multiple Files Using a Map File
With the Aster Database Loader Tool you can load from multiple data files and insert into
multiple tables in a single invocation by passing the --map-file or -m option. We refer to this
as loading with a map file. All tables are loaded in a single database transaction.
When loading a very large amount of data, you may choose to created multiple map files that
each load their data files using a different loader. This can help speed up the process of loading
a large amount of data.
Procedure:
1
Prepare your data files for import. Each file should be formatted as usual for the Aster
Database Loader Tool. All the files you submit in a single running of the Aster Database
Loader Tool must be in the same format.
2
You can optionally pass connection parameters in the map file. Any connection
parameters specified on the command line supercede those in the map file. The
connection parameters you can specify are:
•
dbname - the name of the database
•
username - the name of the Aster Database user
•
password - the password of the Aster Database user
•
loader - the hostname or IP address of the Aster Database loader node
•
force-loader - same as the -f command line option. Instructs the Aster Database
Loader Tool to use the loader node specified with the loader parameter even if the IP
address provided is not known to Aster Database. Note that if this option is specified,
the Aster Database Loader Tool will only try that single IP address and return an error
status if the connection fails for any reason.
•
timeout - same as the -t command line option. Specifies the timeout value in seconds
for Aster Database connection attempts. Default is 30 seconds.
3
Prepare your map file. The map file is a text file containing a set of logical text blocks, each
surrounded by curly braces. Each block represents a file or directory to be loaded. The
format is like this:
{
"loadconfig" :
[
{
"table" : "schema1.targettable1",
"file" : "data/insert1.txt",
"errorlogging" : { "enabled" : false }
},
{
"table" : "schema1.targettable1",
"file" : "data/insert2.txt",
"begin-script" : "input/mapfile/begin-script.sql",
"end-script" : "input/mapfile/end-script.sql",
"errorlogging" : { "enabled" : false }
}
]
}
Aster Client Guide
135
Data Loading
Aster Database Loader Tool
In the above example, we assume the current directory (from which we invoke
ncluster_loader) contains a subdirectory, data, which has two files, insert1.txt and
insert2.txt, and we load these into table targettable1 in schema1. Error logging is
turned of.
Each block in the map file must contain the following required parameter flags and their
values:
•
table specifies the name of the target table. Typically you will schema-qualify the table
name as shown in the example above. You may omit the schema, in which case the
user’s schema search path determines the schema. If your table name is case sensitive,
you must surround the table name with backslash-escaped double quote marks (\"),
like this:
"table" : "schema1.\"TargetTable\"",
•
file specifies the name of the file or directory to be loaded. See “filename” in “Syntax”
on page 126.
•
begin-script and end-script specify scripts to run either before or after the
loading of the file. Both are optional. Each map file entry can have a separate beginscript and end-script. For each map file entry, ncluster_loader will run the
begin-script, load the data from the file, then run the end-script.
The begin-script and end-script require each statement to be on a single line. Do
not include commands that begin or end the transaction in the script files. Here is an
example of a valid script file:
DROP table IF EXISTS foo;
CREATE table foo (id int, sometext varchar(40)) DISTRIBUTE BY HASH
(id);
•
errorlogging introduces a block that specifies how to handle malformed rows in the
input file. Inside the errorlogging block (enclosed in curly braces) you pass the
enabled parameter and, optionally, the parameters discard-errors, label, limit,
schema, and table. These parameters correspond, respectively, to the command-line
flags, --el-enabled, --el-discard-errors, --el-label, --el-limit, -schema, --el-table.
Each block in the map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 127. See “Rules for Passing Arguments in a
Map File”, below.
4
Run the Aster Database Loader Tool, passing the --map-file or -m option and the name
of your map file. Pass additional command line flags as needed, observing the rules set
forth in the preceding paragraph.
Rules for Passing Arguments in a Map File
Observe the following rules when you pass loader arguments in a map file:
136
1
Each block in a map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 127. Observe all rules spelled out there.
2
If you wish to use flags listed in the middle column of that table, you must pass them inside
the map file, and you cannot use their command-line equivalents.
Aster Client Guide
Data Loading
Aster Database Loader Tool
3
Those flags that have no value listed in the middle column can also be used when you’re
loading with a map file, but you must pass them at the command line, instead.
Example map file
The example file reproduced here shows various valid combinations of loading parameters.
Note that the third block in the example (the one inserting into “test3”) instructs the Aster
Database Loader Tool to load all the files it finds in directory “my_data_dir”.
{
"dbname" : "beehive",
"username" : "beeuser",
"password" : "jw8s0F4",
"loader" : "10.51.6.240",
"force-loader" : true,
"timeout" : 5,
"loadconfig" :
[
{
"table" : "test1",
"file" : "data/insert.text",
"errorlogging" : { "enabled" : false }
},
{
"table" : "test2",
"file" : "data/insert.text.n",
"errorlogging" : { "enabled" : false }
},
{
"table" : "test3",
"file" : "my_data_dir/",
"errorlogging" : { "enabled" : false }
},
{
"table" : "test6",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test6"
}
},
{
"table" : "test12",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test12",
"discard-errors" : true
}
},
{
"table" : "test13",
"file" : "data/insert.text.incorrect.pk",
"prefix" : "0",
"columns" : "a,b",
"errorlogging" :
Aster Client Guide
137
Data Loading
Aster Database Loader Tool
{
"enabled" : true,
"label" : "vm_test_12-test13",
"limit" : 100000,
"schema" : "public",
"table" : "nc_errortable_part"
}
}
]
}
Examples
Simple Load Example
Assume the following:
•
File to be loaded: 2010MarchSales_data.txt (assume current working directory)
•
Number of rows: 1000 rows encoded in text format
•
Delimiter: delimited by character "~"
•
Password: beehive
•
Destination table: sales_fact
•
Destination Aster Database: IP address 10.50.25.100
•
Username: assume default
•
Database: assume default
This is what you would type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" sales_fact
2010MarchSales_data.txt
After successful completion, you will see the following output indicating 1000 rows loaded:
1000 tuples were successfully loaded into table customers.
Load from STDIN Example
To load from STDIN, you simply omit the file name/directory name argument. (When
loading from a data file, the last argument to ncluster_loader is the data file name.) For
example, to import our UK sales data and replace the product name “NappyPlus” with
“DiaperPlus” we could use sed and ncluster_loader together like so:
$ sed s/NappyPlus/DiaperPlus/ 2010_UK_sales.csv | ./ncluster_loader -U
mjones -w st4g0l33 sales_fact -c -l 10.51.50.240
Loading from STDIN lets you pipe your data through useful tools like sed, the text stream
editor. Here’s an example of sed at work, ensuring that backslash escape sequences are
properly formatted in the input data before Aster Database Loader sees the data.
Here’s some sample data:
# cat sampleData-3.tsv
5
How often do back-slash characters ('\') appear in your data?
6
And how often do you think they actually disappear: 1 \ ? 2 \ ? 3 \ ?
7
\W\a\y \t\o\o \o\f\t\e\n\! \! \!
138
Aster Client Guide
Data Loading
Aster Database Loader Tool
Here’s how we run Aster Database Loader, piping its input data through sed:
$ cat sampleData-3.tsv \
| sed -e 's_\\_\\\\_g' \
| ncluster_loader -h $QUEEN_IP -d my_db -U beehive -w beehive testo /
dev/stdin
Loading tuples using node '192.168.28.100'.
3 tuples were successfully loaded into table 'test'.
Here are the result rows:
$ act -h $SYSMAN_IP -d my_db -U beehive -w beehive -c 'SELECT * FROM testo ORDER BY id;'
id |
string
----+--------------------------------------------------------------------1 | This is just a line.
5 | How often do back-slash characters ('\') appear in your data?
6 | And how often do you think they actually disappear: 1 \? 2 \? 3 \?
7 | \W\a\y \t\o\o \o\f\t\e\n\! \! \!
(4 rows)
Example with Error Logging
Use the same assumptions as in the previous example, and assume we will log malformed
rows (that is, rows that the loader cannot interpret and therefore cannot load) to a table called
“2010MarchSales_error_table,” tagging each error row with the label
“2010MarchSalesErr”. At the end of the load attempt, the error data will also be copied to
the file, /home/ccrisp/2010MarchSales_error.txt. We’ll set a limit of 100 error rows; if
more than 100 errors are encountered, the load will be cancelled.
To do this:
1
Create the custom error logging table: Run ACT as a user with table creation rights (for
example, one with the catalog_admin role) and type:
CREATE TABLE 2010MarchSales_error_table () INHERITS
(nc_errortable_part);
For more information on creating a custom error logging table, including information on
permissions, .
2
Exit ACT, return to the command line, and type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" --el-enabled --ellabel 2010MarchSalesErr --el-limit 100 --el-table
2010MarchSales_error_table --el-errfile /home/ccrisp/
2010MarchSales_error.txt sales_fact 2010MarchSales_data.txt
For more information on logging malformed rows, see “Error Logging” on page 142.
Hints for Successful Loading
Recommended Character Set Is UTF-8
The default character set for Aster databases is UTF-8, and the Aster team recommends that
you load only UTF-8 formatted data when loading to char, varchar, and text columns.
For the tools you use to prepare files and to connect to Aster, make sure you have set the
default character set to UTF-8. This is particularly important if you are loading data from a
Aster Client Guide
139
Data Loading
Aster Database Loader Tool
Windows-based machine. For example, if you will use an SSH client (e.g., putty) to run
ncluster_loader, make sure you set the SSH client’s default character set to UTF-8.
We recommend that, prior to loading, you convert your text files to UTF-8. For example, if
you’re a Notepad++ user, you can use the command, “Convert to UTF8 without BOM.”
Newline Character
Make sure your data file uses a consistent character to represent newlines. If the file uses \r\n
for newlines, then it should not also use \n for newlines, and vice versa. If your file contains
both UNIX-style \n newlines and Windows-style \r\n newlines, then you must clean the file
before you try to load it. The UNIX command, dos2unix, can be useful for doing this.
Multiple Loader Nodes
The Aster Database Loader Tool supports the use of many Aster Database loader nodes. For
most loading tasks, the queen is sufficient to handle all loading, but for high volume loading,
you can add dedicated loader nodes to your cluster.
To use a loader node, you invoke one or more ncluster_loader instances that will load through
that loader node. You may run many ncluster_loader sessions in parallel against one loader
node, and you may use many loader nodes in parallel (with each node handling loads from a
number of ncluster_loader instances).
To do this, you invoke each ncluster_loader instance with the -l (and optionally -f )
argument to specify the loader node. The required flags are:
•
as always, the --hostname option (-h) provides the queen IP address;
•
the --loader flag (-l) provides the IP address of the desired loader node; and
•
Optionally, the --force-loader flag (-f ) forces the use of the desired loader node.
Loading Parent Child Tables with Inheritance
The --auto-partition option is retained in order to support parent/child tables created
with inheritance. It is not used when working with parent/child tables created with
autopartitioning. Using --auto-partition instructs the Aster Database Loader Tool to
automatically send each row to the right child table during loading. Each row is directed to a
table according to the check constraints you have set up on the child tables.
For example, if you partition your data into daily child tables based on the contents of a
timestamp column, each ultimate child table in your schema will have a CHECK constraint
that specifies what value of timestamp may be loaded into that child table. When you load
data, the autopartitioning feature will route each row to the appropriate child table, based on
its timestamp value.
Use autopartitioning like this:
140
Aster Client Guide
Data Loading
Aster Database Loader Tool
1
Set up the parent-child table schema in your database. On each ultimate child table, write
a CHECK constraint that specifies what data may be loaded into that child table.
Warning! Aster Database does not detect overlapping constraints on peer child tables. As a result, the correct placement of a row during loading can be indeterminate.
Workaround: Take care that the constraints you define do not create overlapping logical partitions. A simple mistake
would be to set up range constraints like this:
CHECK ( ymdh BETWEEN '2005-07-01' AND '2005-08-01' );
CHECK ( ymdh BETWEEN '2005-08-01' AND '2005-09-01' );
In this example, it is not clear in which partition the ymdh value '2005-08-01' resides.
2
3
Prepare your data for loading:
a
Your data input file can contain data values that will end up in many different child
tables.
b
To handle rows that do not fit your partitioning scheme, you can rely on the standard
error logging feature of the Aster Database Loader Tool (see “Error Logging” on
page 142) or create a check constraint that will catch rows that you do not want to
include in your partitions.
Run the Aster Database Loader Tool with the -a or --auto-partition flag.
Using COPY with columnar tables
A loading operation using the Aster Database Loader Tool, COPY, or INSERT can be
expensive when the following conditions exist:
•
the target table uses columnar storage, AND
•
the target table has many logical partitions, AND
•
the loaded data matches many different logical partitions.
In this case, the memory allocated to perform the load may be as large as the amount of source
data. To avoid high memory requirements, it is best to divide a large load into batches. There
are two alternative approaches:
Aster Client Guide
•
Ensure that each batch only loads a small number of logical partitions in the columnar
table. For example, when inserting data into a columnar table with weekly partitions, each
batch may insert data for a single month.
•
Ensure that the size of each batch is a small fraction of system memory available at the
worker nodes. This should only be done if the data being loaded into the columnar table
has a mixture of records matching many different logical partitions. As an example,
suppose that a year's worth of data is being loaded into a columnar fact table with weekly
partitions, and the cluster has four physical worker nodes with 100GB of system memory
in each node. Loading data in 40GB batches will use 10GB memory per physical worker,
which is 10% of the overall available memory.
141
Data Loading
Aster Database Loader Tool
Error Logging
Enabling Load Error Logging
Use the --el-enabled flag (or the errorlogging flag inside a map file) to run the Aster
Database Loader Tool in a mode in which it tolerates poorly formatted input rows and logs
each bad row to a table. This differs from Aster Database Loader’s normal mode of running:
•
Running normally, Aster Database Loader aborts the load immediately if it encounters a
bad input row, and it does not log the malformed input row to a table.
•
Running in --el-enabled mode, Aster Database Loader logs each malformed input row
(that is, any row it cannot interpret for loading or cannot load due to datatype mismatch
or check constraint violation) to an error table and continues to load the remaining rows
in the load job. We refer to this as “error logging.”
The --el-enabled flag is a master flag that operates with a set of sub-flags (--el-discarderrors, --el-errfile, --el-label, --el-limit, --el-schema, and --el-table) that
fine-tune your handling of malformed rows. To use any of the sub-flags, you must first have
specified the --el-enabled flag. (If you’re using a map file, the syntax is different. The
master flag is errorlogging, and the sub-flags are discard-errors, errfile, label,
limit, schema, and table.)
The --el-discard-errors flag discards all malformed rows, the --el-label tags failed
row data, the --el-limit flag sets a maximum number of allowed failed rows for the job, and
the --el-table flag specifies a custom error logging table. See “Argument Flags” on page 127
for explanations. See “Example with Error Logging” on page 139 for example usage.
To perform error logging, the Aster Database Loader Tool relies on the error handling features
of the Aster Database COPY command in SQL.
Load Error Logging Tables
By default, Aster Database logs error rows to one of its default error logging tables, based on
type of the target table: malformed rows for distributed tables go into table
nc_errortable_part table, and malformed rows for replicated tables go into the
nc_errortable_repl table. Alternatively, you can create your own error logging tables. You
can perform SELECT, UPDATE, and DELETE operations on the error log tables, and you can
inherit from the default error logging tables to create your own error logging tables.
Statistics About Logged Errors
To see the number of rows that loaded or failed to load, query the load error statistics tables,
nc_all_errorlogging_stats and nc_user_errorlogging_stats. Each loading
command generates a row in the tables. For a given transaction, the totalcount, goodcount,
and malformedcount columns show the total number of rows you tried to load, the number
of rows that successfully loaded, and the number of rows not loaded, respectively.
142
Aster Client Guide
Data Loading
Troubleshoot Loading
Troubleshoot Loading
Running Multiple Loaders Concurrently
When running multiple ncluster_loader instances concurrently, eventually Aster Database will
throw errors due to physical limitations imposed by system resources. Due to this, you should
not run more than ten ncluster_loader instances concurrently.
Load stalls upon cancellation or failure encountered during a load
The Aster Database Loader Tool can take a lot of time to finish after a failure is detected.
Likewise, if a Control+C is issued, the loader can take several minutes to cancel completely.
Load fails on UNIQUE or PRIMARY KEY violation
A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to
abort. So if your load is failing, check your data and the schema of the table you are loading
into for these violations.
Invalid Input Syntax Error
If you encounter an error of type:
"Error: Invalid input syntax on partition key for..."
then there may be a problem with the character encoding of your input data. Please check the
encoding of your input data and the datatypes of your destination columns, fix any
mismatches you find, and retry the load.
Single quote character must be escaped when using the -q option
If you encounter an error like:
"ERROR:
extra data after last expected column"
you may have encountered a problem with command line parsing. If you used a single quote
delimiter in your CSV input file, and the input data includes a comma within one of the text
strings to be loaded, you will see this error.
To correct this, you will need to escape the single quote character when specifying it using the
-q option on the command line as -q"\'".
Here is an example:
Assume the file test.txt, which we are attempting to load into table table1 includes the
following input data row:
1, 25, "Some text with a, comma in it."
The following will not load, returning the "Error: extra data after last expected
column" error:
ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d
beehive -c -q"'" table1 test.txt
Aster Client Guide
143
Data Loading
Troubleshoot Loading
The following will load:
ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d
beehive -c -q"\'" table1 test.txt
Using the -C option with uppercase or special characters
When using the -C option where the column list has any uppercase or special characters, you
must put the column list in double quotes. On Windows, this requires escaping the double
quotes:
Example on Linux:
./ncluster_loader -h 10.51.150.100 -l 10.51.150.240 -w db_superuser -U
db_superuser -c -D , -C '"ID","NAME","$value"' test2 test2.csv
On Windows, when using the -C option where the column list has any uppercase or special
characters, you must put the column list in escaped double quotes.
Example on Windows:
ncluster_loader.exe -h 10.51.150.100 -w db_superuser -U db_superuser -c
-C '\"ID\",\"NAME\",\"$value\"' test2 test2.csv
Uppercase characters are passed as lowercase if not escape quoted
If your data contains a table or schema with a capital letter in its name, you must escape quote
the table or schema name. If you do not do this, all lower case letters are assumed. Because of
this, the data will load into a schema or table having the same name all in lower case, if it
exists.
For example, if you specify the table Test.Try, Test.try, test.Try or test.try, the data will always
be loaded into test.try unless you specify escape characters with enclosed quotes for the table
name parameter to ncluster_loader as in this example:
# /home/beehive/clients/ncluster_loader -U db_superuser -w db_superuser
-c \"Test\".\"Try\" file001
Loading tuples using node 'localhost'.
4 tuples were successfully loaded into table '"Test"."Try"'.
This example loads the data in the expected table (Test.Try).
144
Aster Client Guide
CHAPTER 6
•
Export and Load Tools
In addition to Aster Database Loader (described in “Aster Database Loader Tool” on
page 125) Aster Database provides the Aster Database Export Tool.
Aster Database Export Tool
The Aster Database Export Tool, ncluster_export, is a command-line application for bulkexporting data from Aster Database to a file or to STDOUT. It provides an alternative to the
SQL COPY TO statement. Each invocation of the Aster Database Export Tool is equivalent to
a single invocation of the COPY TO statement. Each invocation is a single database transaction
that exports the contents of the specified table into the specified file or STDOUT.
Synopsis
To run the Aster Database Export Tool, you type:
$ ./ncluster_export [arguments] [schemaname.]tablename [ filename ]
You can also pipe the results through a standard UNIX command such as gzip. For example:
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb mytable
| gzip -c > mytable.gz
In the synopsis above, the arguments are:
•
schemaname is the optional name of the source table’s schema. If no schema name is
provided, Aster Database will search the schemas listed in your current schema search
path.
•
tablename is the name of the source table. See “Case-Sensitive Handling for Table
Names” on page 146 if you wish to have Aster Database evaluate table names in a casesensitive manner. To export from multiple tables, see “Exporting from Multiple Tables” on
page 146.
•
filename indicates the name of the file that will receive the exported data. If you don’t
supply a file or directory name argument, Aster Database Export assumes you want to
export to STDOUT. If the filename contains spaces, make sure you enclose it in double
quotes.
Aster Client Guide
145
Export and Load Tools
Aster Database Export Tool
•
arguments are the command-line flags that control how the exporter runs. The flags are
explained in “Argument Flags for Exporter” on page 147, or you can display the help by
typing:
$ ncluster_export -?
Case-Sensitive Handling for Table Names
To force Aster Database Export to treat your table and schema names in a case-sensitive
manner, you must surround the schema and table names in double quote marks when you
invoke ncluster_export. To pass a double quote mark to Aster Database Export, you must
prefix it with the escape character, which is a backslash. That means you type a double quote
mark like so: \"
To do this, surround the entire schemaname.tablename string in a single pair of double
quotation marks.
For example, assuming you want to export to table "t" in schema "S", then you would invoke
Aster Database Export in this form:
ncluster_export.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"S\".\"t\"
mydata.txt
Exporting from Multiple Tables
With ncluster_export you can export from multiple tables in a single invocation by creating a
map file of tables to be exported and passing the --map-file or -m option with the filename.
All tables are exported within a single database transaction. To export from multiple tables,
perform the following steps:
1
Prepare your map file. The map file is a text file containing a set of logical text blocks, each
surrounded by curly braces. Each block represents a table to be exported. The format is as
follows:
{
"exportconfig" :
[
{
"table" : "exporttable1",
"fileprefix" : "path/to/dir/file"
},
{
"table" : "exporttable2",
"fileprefix" : "path/to/dir/prefix",
"maxtuplesperfile" : 100,
"columns" : "a,b"
}
]
}
Each block must contain the following required parameters:
•
“table” specifies the name of the table to export.
•
“fileprefix” specifies the output file / file prefix to be used.
Each block can contain the following optional parameters:
146
Aster Client Guide
Export and Load Tools
Aster Database Export Tool
2
•
“maxtuplesperfile” can be used to specify how many records will be put in each output
file. This allows outputting multiple files for a single table, with the specified number
of records in each.
•
“columns” is a comma-separated list of the columns to be exported.
Run ncluster_export, passing the --map-file or -m option and the name of your map
file.
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb -m
"path/to/dir/mapfile"
Argument Flags for Exporter
Aster Database Export accepts the following flags.
Table 6 - 1: Aster Database Export Flags
Aster Client Guide
Column
Type
-? [ --help ]
Shows the online help.
-B [ --begin-script ] arg
Specifies the qualified path of the file containing SQL
commands that should be executed when the
transaction starts, i.e. immediately after the begin
command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the begin script should be on a
separate line. Do not include BEGIN or END
statements.
-c [ --csv ]
Exports the output files in CSV format (the default is to
use Text format).
-C [ --columns ] arg
An optional comma-separated list of columns to be
exported from the source table (the default is to export
all columns).
-d [ --dbname ] arg
Specifies the name of the database to connect to (the
default is 'beehive').
-D [ --delimiter ] arg
Specifies the delimiter character to use when exporting
data (must be a string that represents a valid single
character, such as 'd' or '\n'). The default is a tab
character ('\t') in text mode, a comma (',') in CSV
mode.
-E [ --end-script ] arg
Specifies the qualified path of the file containing SQL
commands that should be executed when the
transaction finishes, i.e. immediately before the end
command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the end script should be on a separate
line. Do not include BEGIN or END statements.
147
Export and Load Tools
Aster Database Export Tool
Table 6 - 1: Aster Database Export Flags (continued)
Column
Type
-e [ --escape ] arg
Specifies the escape character for CSV output (must be a
string that represents a valid single character, such as 'd'
or '\n'). The default is the quote value - double-quote by
default. This option is only valid when CSV mode is
specified.
-f [ --force-loader ]
Instructs ncluster_export to use the loader node
specified with the '-l/--loader' option even if the IP
address provided is not known to Aster Database.
NOTE: ncluster_export will only try that single IP
address and return an error status if the connection fails
for any reason.
filename (Not a flag; you pass
the file or directory name itself!)
filename indicates the name of the file that will receive
-h [ --hostname ] arg
Specifies the hostname or IP address of the machine on
which Aster Database is running. Default is 'localhost'.
-l [ --loader ] arg
Preferred loader IP address. If a value is provided,
ncluster_export will try to export through this IP
address before trying any other loader node. NOTE:
ncluster_export expects this IP address to be known by
Aster Database and will simply ignore the address
provided if this is not the case. To change this behavior
one can use the '-f/--force-loader' option.
-m [ --map-file ] arg
Specifies the name of the file containing mappings of
the tables to be exported. This option allows export of
multiple tables within the same transaction. For details
about the format used for the map file, see “Exporting
from Multiple Tables” on page 146.
-M [ --max-tuples-perfile ] arg
Specifies how many records are exported to a single file.
If the total number of records exceeds this number
multiple output files with the suffix '_N' are produced,
where N is an integer.
-n [ --null ] arg
Specifies a string that represents a NULL value. The
default is '\N' in text mode and an empty value with no
quotes in CSV mode. See also “--null-backslashescapes” on page 148.
--null-backslash-escapes
Indicates that backslashes in the '-n/--null' flag should be
treated as special characters, which will be processed
according to the default rules for escaped strings.
the exported data. If you don’t supply a file or directory
name argument, Aster Database Export assumes you
want to export to STDOUT. If the filename contains
spaces, make sure you enclose it in double quotes.
You should use this flag whenever the null string
contains a backslash, to ensure compatibility with future
versions of Aster Database.
148
Aster Client Guide
Export and Load Tools
Aster Database Export Tool
Table 6 - 1: Aster Database Export Flags (continued)
Column
Type
-p [ --port ] arg
Specifies the TCP port on which Aster Database is
listening for connections. The default is port 2406.
-q [ --quote ] arg
Specifies the quote character for CSV output (must be a
string that represents a valid single character, such as 'd'
or '\n'). The default is the double-quote. This option is
only valid if CSV mode is specified.
schemaname (Not a flag; you pass
schemaname is the optional name of the source table’s
schema. If no schema name is provided, Aster Database
will search the schemas listed in your current schema
search path.
the schema name itself!)
-t [ --timeout ] arg
Specifies the timeout value (in seconds) to use when
connecting to Aster Database (default is 30).
tablename (Not a flag; you pass
the table name itself!)
tablename is the name of the source table. See “CaseSensitive Handling for Table Names” on page 146 if you
wish to have Aster Database evaluate table names in a
case-sensitive manner. To export from multiple tables,
see “Exporting from Multiple Tables” on page 146.
-U [ --username ] arg
Connects to the database with the specified username
instead of the default ('beehive'). (You must have
permission to do so, of course.)
-V [ --version ]
Prints the version of ncluster_export and exit.
-w [ --password ] arg
Connects to the database with the specified password
(as opposed to providing a password at the prompt).
-W [ --password-prompt ]
Forces ncluster_export to prompt for a password before
connection to the database.
Installing Aster Database Export
You can run ncluster_export directly on the queen or install it on your workstation. On the
queen, it is located by default at /home/beehive/clients/ncluster_export.
Supported Platforms
For a list of supported operating systems for the Aster Database drivers and utilities, see the
Aster Database Drivers and Utilities Guide for your version of Aster Database.
On Linux, ncluster_export requires glibc 2.7 or later.
If you are running on AIX, see “AIX Client Dependent Libaries” on page 44.
Procedure
To install the Aster Database Export Tool on a machine other than the queen:
Aster Client Guide
149
Export and Load Tools
Aster Database Export Tool
1
To obtain the Aster Database Export Tool package for your client operating system,
download it from one of these places onto your client machine in the directory where you
will install it:
•
To get the newest package, download it from http://downloads.teradata.com/download/
tools
•
2
150
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
Set permissions to make the files executable.
Aster Client Guide
Index
A
about this book 10
ACT
check version of ACT 19
command history 32
command reference 30
command-line flag reference 18
file management commands 32
installing 11
launching 16
launching on Linux or Solaris 17
launching on Mac 17
launching on Windows 17
list all databases 34
list all tables in Aster Database 33
parameter, setting with \set 31
parameters 18
parameters at SQL prompt 30
tab completion 30
transpose columns and rows in output 35
using ACT 24
ACT commands 30
command-line options 18
SQL-prompt options 30
Active Directory authentication
SSL connections and 84
AIX
configure ODBC Driver Manager 50
install ODBC driver 50
ANALYZE
bulk loading and 124
Aster Data, about 10
Aster Database
checking version of ACT 19
connecting, overview 43
Aster Database configuration settings
SQL behavior 71
Aster Database Export: See ncluster_export.
Aster support portal 10
auto-analyze flag in batch loading 132
auto-partition flag in batch loading 127
autopartitioning
ncluster_loader and 140
B
backslash
Aster Client Guide
adding to input stream with sed 138
allow backslash escapes 71
backslash escape sequences 138
begin-script argument during batch load 136
begin-script flag in batch loading 127
best practices
loading 121
bulk load 121
autopartition 140
end-script 129
from many files 135
logging bad rows during import 142
map file 135
ncluster_loader 125
bulk loading
arguments 127
flags 127
map file: supported flags 127
parameters 127
bytea
handling bytea data in ODBC 57
C
certificate 90
for ODBC connection 90
character set for loading 139
character set, default 44
characters, special 44
loading 139
child table
automatically route data into 140
client 11
installing ACT client 11
recommended client settings 44
client settings, recommended 44
client software versions 19
client tools 11
client-side cursors in JDBC 67
column
transpose columns and rows 35
column alignment of query output 34
columns argument during batch load 128
command buffer 32
command history 32
send to file 32
command reference
151
ACT command-line flags 18
ACT SQL prompt 30
command-line (in SQL) commands for ACT 30
command-line options for ACT 18
commands
ACT command-line flags 18
ACT commands at SQL prompt 30
configuration
queen system parameters 85
configuration settings
SQL behavior 71
connect 43
JDBC driver 59
Microsoft .NET, drivers for 97
MicroStrategy 93
ODBC driver 44
connecting 43
overview 43
through JDBC 62
conventions 9
coordinator.cfg 85
COPY
in ncluster_loader 125
copyright 10
csv flag in batch loading 127
cursor
ACT, cursors in 26
distributed 67
JDBC, cursors in 67
customer support 10
customizing SQL behavior 71
D
data
autopartition during load 140
bulk load 121
encrypting data traffic 83
data export 145
data loading 121
database
list all tables 33
list the databases in a cluster 34
database drivers 43
JDBC 59
Microsoft .NET, drivers for 97
ODBC 44
ODBC for Perl scripts 55
date
date format in bulk loading 128
date of publication 10
date-style flag in batch loading 128
dbname flag in batch loading 128
delimiter flag in batch loading 128
152
delimiters
setting in ACT 34
discard-errors flag in batch loading 129
distributed cursors 67
documentation conventions 9
documentation version and updates 10
documentation, about 9
dot-NET data provider 97
Driver Manager
configuration 50
ODBC 50
drivers 43
JDBC 59
Microsoft .NET, drivers for 97
ODBC 44
ODBC for Perl scripts 55
E
E prefix 71
edition 10
el-enabled flag in batch loading 129
ELT 121
enable_quoted_identifiers 71
encryption 77
of query results 83
SSL for ODBC 77
end-script argument during batch load 136
end-script flag in batch loading 129
error file flag in batch loading 129
error limit in bulk loading 129
error logging 142
loading, log of errors for 142
tables to capture load errors, default 142
error logging file 129
error logging table, naming 130
errorlogging argument during batch load 136
escape character 128
allow backslash escapes 71
bulk loading and 128
ncluster_loader and 128
escape flag in batch loading 128
escape sequence
adding to input stream with sed 138
ETL 121
SSIS and 99
expanded output mode 35
export 145
Aster Database export tool 145
JDBC driver 59
ODBC driver 44
source table, case-sensitive name 146
tools for 145
export data 145
Aster Client Guide
sample test program 69
export tool 145
F
L
FETCH_COUNT 26
FETCH_LIMIT 28
fetch-count 26
fetch-limit 28
field separator, setting in ACT 34
file
redirect command history to file 32
redirect query history to file 32
redirect query results to file 32
file argument during batch load 136
file input to ACT 32
with \i 32
with -f 19
force-loader flag for bulk loading 130
format
dates in bulk loads 128
label argument during batch load 136
label for data loading errors 129
LIMIT
FETCH_LIMIT as alternative to 28
limit argument during batch load 136
limit rows returned at a time 26
limit total rows returned per query 28
line break character 140
list all tables 33
list databases command 34
load 121
autopartition 140
from many files 135
from SDTIN 138
logging bad rows during import 142
map file 135
load data 121
other tools 145
loader
arguments 127
Aster Database loader tool 125
destination table 126
destination table, case-sensitive name 126
flags 127
map file: supported flags 127
parallel loading with 140
parameters 127
specifying a dedicated loader node 140
loader flag for bulk loading 130
loader node 121
parallel loading with 140
loader tool 125
loading 121
arguments 127
Aster Database Export tool 145
Aster Database Loader tool 125
best practices 121
character set for loading 139
flags 127
from SDTIN 138
handling nulls when loading data 131
JDBC driver 59
map file: supported flags 127
ODBC driver 44
parameters 127
SSIS and 99
troubleshooting 140
troubleshooting problems encountered when loading 143
loading errors 142
log errors 142
G
get latest documentation 10
group
list all groups in Aster Database 34
H
help 10
help for ncluster_loader 130
history in ACT 32
hostname flag for bulk loading 130
I
import
from many files 135
logging bad rows during import 142
ncluster_loader 125
index
list all indexes in Aster Database 33
install
ACT client 11
Aster Database Loader Tool 133
J
Java
JDBC statements 90
JDBC 59
connecting through JDBC 62
cursors in 67
driver 59
how to write applications that use JDBC 90
query example 91
Aster Client Guide
153
custom error logging tables 142
loading
errors, logging 142
SSL settings 79
output
transpose columns and rows 35
M
P
malformed rows during load 142
map file 135
supported flags 127
map file for ncluster_loader 135
map-file flag for bulk loading 131
memory usage in ACT: limit with FETCH_COUNT paging of
query results 26
Microsoft .NET, drivers for 97
Microsoft SSIS 99
MicroStrategy, connections for 93
parameter
setting queen system parameters 85
settings for ACT 31
partitioning
autopartition during load 140
performance tuning
improve ACT performance with FETCH_LIMIT 28
limit ACT memory use with FETCH_COUNT 26
server-side cursors in ACT 26
Perl scripts, ODBC driver for 55
pipe to ncluster_loader 138
pivot column and row output 35
port flag for bulk loading 131
portal 10
preferences
SQL settings 71
prefix argument during batch load 131
PreparedStatement in JDBC 90
pre-process data with sed 138
N
nc_errortable_part 142
nc_errortable_repl 142
ncluster_export 145
source table, case-sensitive name 146
ncluster_loader 125
arguments 127
character set for loading 139
date format 128
destination table 126
destination table, case-sensitive name 126
flags 127
installing 133
loading from many files 135
loading to multiple tables 135
logging bad rows during import 142
map file: supported flags 127
parameters 127
.NET Data Provider 97
newline character 140
null
handling nulls when loading data 131
null flag for bulk loading 131
O
ODBC 44
bytea handling 57
driver 44
Driver Manager 50
Driver Manager configuration 50
ODBC driver
for MicroStrategy 93
installing for use with Perl scripts 55
installing on AIX 50
installing on Linux 48
installing on Windows 46
154
Q
queen
system parameters, setting 85
query
transpose results 35
typing in ACT 24
via Java JDBC calls 90
query buffer 32
send to file 32
query history 32
send to file 32
query results 32
send to file 32
query tool
installing ACT client 11
quiet mode of ACT 21
quote character flag for bulk loading 131
quoted identifier
enable in Aster Database 71
R
recommended client settings 44
recommended loading settings 139
redirect command history to file 32
redirect query history to file 32
redirect query results to file 32
reporting tools
Aster Client Guide
MicroStrategy 93
results 32
send to file 32
row
transpose columns and rows 35
running SQL scripts 32
with \i 32
with -f 19
S
sample code
Microsoft dot.NET example 99
schema
list all schemas in Aster Database 34
schema argument during batch load 136
scripts 32
run SQL script with \i 32
run SQL script with -f 19
security
SSL for ODBC 77
sed
for backslash sequences 138
sed, using with loader 138
send command history to file 32
send query history to file 32
send query results to file 32
server-side cursors
in ACT 26
set command in ACT 31
settings
enable_backslash_escapes 71
enable_quoted_identifiers 71
SQL settings 71
SSL connection settings 79
SQL
customizing SQL behavior 71
parameters that affect SQL command interaction 30
parameters that set ACT client behavior 18
parameters that set SQL behavior 71
redirect query results to file 32
scripts, running with \i 32
scripts, running with -f 19
SQL commands
utility commands in ACT 30
SQL COPY 125
SQL prompt commands 30
SQL prompt, using in ACT 24
SQL tool, installing ACT client 11
SSH 17
SSH client 17
SSIS 99
SSL 77
with Active Directory authentication 84
Aster Client Guide
SSL for ODBC 77
certificates for 90
client settings 79
common configuration scenarios 80
encryption of query results 83
how to set up on client 85
how to set up on queen 85
queen settings 79
reference to Aster Database SSL settings 79
SSO for ODBC connections 84
Statement in JDBC 90
STDIN
loading data from 138
support 10
system tables
nc_errortable_part 142
nc_errortable_repl 142
T
tab completion 30
table
list all tables in Aster Database 33
table argument during batch load 136
table argument inside errorlogging block 136
TD Wallet 73
commands 74
configure 74
install 74
technical support 10
Teradata Wallet 73
terminal
installing ACT client 11
tools 11
MicroStrategy 93
SSH client 17
troubleshooting
load attempt fails 140
loading, troubleshooting for 143
TRUNCATE
before bulk loading 132
truncate-table flag for bulk loading 132
typeface conventions 9
U
umlauts 44
updated documentation 10
URL 10
Aster Support URL 10
user
list all users in Aster Database 34
UTF-8
loading 139
recommended client settings 44
155
utilities 11
JDBC driver 59
Microsoft .NET, drivers for 97
ncluster_export 145
ncluster_loader 125
ODBC driver 44
V
VACUUM
after bulk loading 132
vacuum-table flag for bulk loading 132
version
check ACT version 19
documentation version 10
ncluster_loader version 132
W
workaround
certificate, missing self-signed cert 78
X
x command 35
156
Aster Client Guide