Download Dimension Build - dbmanagement.info

COGNOS Decision Stream
Cognos CoE
Internal and Confidential
Agenda
• Getting Started
• Components of DecisionStream
• Definitions
• DecisionStream Catalog
• Database Connections
• Fact Builds
• Dimension Builds
• JobStreams
• SQLTXT Designer
• SQL Term
• Command Line Interface
22 May 2017
Modules
1. Getting Started
2. Create a Catalog
3. Fact builds
4. Data Delivery Modules
5. Dimension Builds
6. Job streams
7. SQL Text Designer
8. Command Line Interface
9. Definitions
22 May 2017
Module 1
Getting Started
22 May 2017
What is DecisionStream?
• The primary purpose of DecisionStream is to create integrated data marts by:
– extracting operational data from multiple sources
– merging and transforming the data to facilitate reporting and analysis
– delivering transformed data into coordinated data marts
22 May 2017
The Purpose of DecisionStream
Transformations
Data
Data
OLTP
OLTP
OLTP
OLTP
Data
Integrated
+
x/y
-
Data Marts
OLTP
22 May 2017
Decision Stream Architecture
Platforms
Windows
NT,95,98,
2000
Desktop
Design Client
API
Scripts
DecisionStream Process Metadata
NT or UNIX
Server
HPUX
AIX
Solaris
IBM AIX 4.2
COMPAQ TRU64
Operational
Sources
DB2
Oracle
SQL Server
Informix
ODBC
Sybase
Text
MetaData
Targets
Impromptu
Transformer
Architect
Custom
DBMS
Flat Files
Microsoft SSAS
Essbase
ERP
Server Engine
22 May 2017
How DecisionStream Creates Data Marts
Dim
Data
Sources
Dimensional
Framework
Product
Time
Location
1
Sales
3
5b.
Dimension
Delivery
Data
Source
5a.
2
Fact
Delivery
5c.
Data
Source
MetaData
Delivery
4
22 May 2017
DecisionStream Interface
22 May 2017
Interface Objects
• The core elements in a DecisionStream catalog are:
– Builds
– DataStreams
– Transformation Models
– Delivery Modules
22 May 2017
Interface Objects (Cont’d)
• The core elements in a DecisionStream catalog are:
– JobStreams
– Dimensions
• Hierarchies and Lookups
• Templates
– Connections
– Functions
22 May 2017
DecisionStream Tools
22 May 2017
Module 2
Create a Catalog
22 May 2017
What is a Catalog?
A catalog is a container for DecisionStream metadata.
22 May 2017
DecisionStream Catalog
• A DecisionStream Catalog provides a central repository for the information that
defines how DecisionStream extracts, transforms, and delivers data.
• You can store a DecisionStream Catalog in either a dedicated database or a
database that you use for other purposes, such as a target data mart.
• A Catalog consists of a set of data tables that can reside in any database for which
DecisionStream has create permissions. You can create only one Catalog per
database.
22 May 2017
DecisionStream Catalog
Diagram illustrates
• How DecisionStream extracts and
transforms data from the data sources
and how it delivers data to the target
marts
• The flow of data from the data sources
to the target data marts
22 May 2017
Shared Items in a Catalog
• Library items can be used throughout the
catalog in a variety of different builds and
JobStreams.
• A catalog library consists of:
– Dimensions
– Connections
– Functions
22 May 2017
DecisionStream Catalog
Create a Catalog
You can create a Catalog in the DS_New sample database.
Steps
1. From the File menu, click New Catalog. If you are prompted to disconnect from the
current Catalog, click OK.
The New Catalog dialog box appears.
2. In the Name box, type MyCatalog and then click Next.
3. In the left pane, click ODBC.
4. In the Data Source Name box, click DS_New.
5. Click Finish.
DecisionStream creates the Catalog tables and opens the new Catalog.
6. From the File menu, click Save Catalog.
22 May 2017
Backup and Restoration of a Catalog
• Backup files are useful for recovery and emergency situations.
• Write the catalog tables to an ASCII text file for later recovery.
22 May 2017
Database Connections
Creating an ODBC Connection
Steps to define connections to ODBC databases set up for the Great Outdoors Company.
Steps
1. Open the MyCatalog catalog that you created in Chapter 2.
2. In the Library, right-click Connections, then click Insert Connection.
3. In the Alias box, type GO_Sales
4. Click the Connection Details tab.
5. In the left pane, click ODBC.
6. In the Data Source Name box, click GOScer3.
7. Click OK to close the Connection Properties dialog box.
22 May 2017
Database Connections
Creating a connection to SQLTXT Database
Steps
1. In the Library, click the Connections folder.
2. From the Insert menu, click Library, and then click Connection.
3. In the Alias box, type DS_Sources
4. Click the Connection Details tab.
5. In the left pane, click SQLTXT.
6. In the Definition File box, click Browse, then select the DS_Sources.def file in the
Cognos samples directory.
8. Click OK to close the Connection Properties dialog box.
22 May 2017
Database Connections
Creating a connection to Oracle Database
Steps
1. In the Library, click the Connections folder.
2. From the Insert menu, click Library, and then click Connection.
3. In the Alias box, type DS_OracleSrc
4. Click the Connection Details tab.
5. In the left pane, click Oracle SQL*Net.
6. Enter the values for User Name, Password and SQL*Net Service.
8. Click OK to close the Connection Properties dialog box.
22 May 2017
Module 3
Fact Builds
22 May 2017
Fact Builds
A DecisionStream fact build specifies how to extract, transform, and deliver data.
Data may come from multiple data sources,
Each of which has its own naming and storage conventions.
A fact build can
• acquire data from one or more data sources
• merge and clean the source data
• aggregate the data with reference to the dimensional framework
• deliver the fact and dimension data to one or more data marts
• deliver metadata to OLAP systems and Cognos BI applications
22 May 2017
The Fact Build Process
Structure
Data
Sources
Dimensional
Framework
Deliver
10100101
…
Transaction
Data
Sources
Acquire
Star Join
Merge
Transform
Aggregate
Partition
Filter
22 May 2017
Fact Build Data Elements
Dimension
A dimension key or data that is used to qualify a
measure.
Measure
Attribute
Quantitative data or values that can be aggregated.
These are usually Key Performance Indicators
(KPIs).
Additional information that is not a dimension
element or measure but that may be of interest.
Derivation
A calculation or transformation.
22 May 2017
Fact Build Wizard
The Fact Build Wizard guides you step-by-step through the process of
creating a new fact build.
22 May 2017
Types of Fact Builds
Fact build types provide default settings for the delivery of data marts.
22 May 2017
The Fact Build Wizard: Add a Data Source
A data source for the fact build is chosen from the available connections. This source
contains the transactional data to be transformed.
22 May 2017
The Fact Build Wizard: Create the SQL Data Source
Tables and columns
to be read from
SQL statement
created
22 May 2017
The Fact Build Wizard: Map to Fact Build Elements
Columns returned from
source data by SQL
Elements of the
Transformation Model
22 May 2017
The Fact Build Wizard: Modify Attributes
The elements of the Transformation Model are created automatically by the Fact
Build Wizard but may need to be modified.
Click here to modify
the types of elements
Elements of the
Transformation Model
22 May 2017
The Fact Build Wizard: Link to the Dimensional Framework
Dimension elements are related to reference structures.
22 May 2017
The Fact Build Wizard: Fact Data Delivery
22 May 2017
Fact Build Visualization
Dimensions
associated with
the build
Transactional
data sources
Fact and
Dimension
tables
Metadata delivery to
Impromptu, PowerPlay,
and Architect
22 May 2017
DataStream Mapping Visualization
You can visualize the mapping of the SQL columns to the build elements by
clicking the DataStream tab.
22 May 2017
Transformation Model Visualization
Use the Transformation Model tab to view the linkage of the dimension
elements to the Dimensional Framework.
Mapped Reference Structures
22 May 2017
Fact Delivery Visualization
By clicking the Fact Delivery tab, you can view the mapping of the elements
to the target fact table.
22 May 2017
Delivery Modules
• The three delivery module types are:
– fact
– dimension
– metadata
22 May 2017
Fact Builds
Steps to Execute the Fact Build
1. Right-click the BIMart fact
build, and then click Execute.
The Execute Build dialog box
appears.
2. In the Execution mode box,
click Normal, and then click OK.
DecisionStream executes the
Fact build in a command
window, writing progress
messages to the screen and
execution log.
3. When the fact build execution
Is complete, press Enter to close
The command window.
22 May 2017
Run a Fact Build
22 May 2017
Create a Fact Build Manually
1 Add a build
3 Add and define
the build elements
2 Define the source
4 Map the elements to the columns
SQL columns
Build elements
22 May 2017
Add a Dimension Element to the Fact Build
Insert
Associate and Set
Select to roll up to
the higher level(s)
Select to include all
records to avoid
rejected data
You have the option of adding
the dimension element to
deliveries after the fact table
has been created
Select to use the
surrogate, if one is
available, as a
foreign key
22 May 2017
Add a Measure to a Build
Insert and name it
Specify a merge behavior
Specify an aggregation method
22 May 2017
What is a Derivation?
• Derivation is an element that DecisionStream calculates rather than obtains directly from the data source.
• It derives from any build element.
• It is an output-only element.
ProductRevenue = Quantity * UnitPrice
Quantity * UnitPrice
22 May 2017
When to Use a Derivation?
• Use a derivation when:
– a calculation needs to be stored in the data mart to be shared by users
– the calculation is very complex and open to misinterpretation
– certain key performance indicators (KPIs) must be derived
– transformation is required
22 May 2017
Operators for Derivations
Mathematical:
– ( )
– +
– – *
– /
Logical:
=, !=
>=, <=
>, <, <>
IS
AND, OR
[NOT] LIKE
[NOT] BETWEEN
[NOT] IN
NOT ( ……….. )
22 May 2017
General Functions for Derivations
Function Type
Examples
Text
Char(), Concat(), Trim(), Length(),...
Mathematical
Abs(), Floor(), Trunc(), Round(), Mod()
Logical
If(), IfNull()
Type Conversion
ToChar(), ToDate(), ToInteger()
Date
AddToDate(), SysDate(), DaysBetween()
Control
RowsUpdated(), Exit(), SQL(), LogMsg(),..
Member
Member(), Level(), IsAncestor()
22 May 2017
Control Statements
Statements
Assignment
Return
if…
then…
else…
while…
do…
case…
<block>
Syntax
<va ria ble> := <expression>;
return <expression>;
if<expression>
then<block>
else<block>
while <expression>
do <block>
ca se <expression> of
begin
<va lue1>: <block>
defa ult: <block>
end
begin<sta tement>;end
22 May 2017
How to Define a Derivation
Insert
Calculate
Test
22 May 2017
When to Calculate a Derivation


By default , a derivation is calculated after aggregation.
You can choose to calculate the derivation before aggregation.
Select the Calculate at Source box if you want to
calculate first and then aggregate.
Select an aggregation method
AverageRevenue
Quantity * UnitCost
Revenue
(Quantity*UnitCost)
MEASURE
MEASURE
DERIVATION
AVGRevenue
AGGREGATION
22 May 2017
Module 4
Data Delivery Modules
22 May 2017
Delivery Process to a Single Data Mart
Fact Build
Delivery Modules
Delivery Targets
Dimension
Delivery
Fact
Delivery
Data Mart
Metadata
Delivery
22 May 2017
Fact Delivery Modules
There are two groups of fact delivery modules.
Table delivery modules
Nontable delivery modules
•Bulk Copy (BCP)
•Text File
•DB2 LOAD
•DataStream
•Informix LOAD
•Essbase Direct Feed
•ORACLE SQL *Loader
•Essbase File Feed
•Red Brick Loader (TMU)
•Relational Table
22 May 2017
Fact Delivery: Relational Table Delivery Module
Target database
Target table
Indexing options
Refresh Type:
• APPEND
• TRUNCATE
• REPLACE
Elements
subscription
for delivery
• UPDATE/INSERT
• UPDATE
Commit Interval:
Key option
1
..
100 commit
…
200 commit
…
…
1000
22 May 2017
Fact Delivery: Text File Delivery Module
Elements subscription for delivery
Output Filename
full directory path and file
name or just file name.
Line/Field Delimiter
NL(new line) TAB, COMMA,
SPACE, NONE(no delimiter),
or a character string.
22 May 2017
Dimension Delivery in a Fact Build
• In general you should not deliver dimensional data in a fact build. However, there
are four exceptions to this rule:
– a demo or proof of concept
– data mart delivery from a warehouse
– dimensional data available only in fact data
– metadata delivery
22 May 2017
Dimension Delivery Module
Type or select a name for dimension table
Create or select a template for delivery
Add Relational Table
Add Dimension Table
Template
Attributes
Map the data set columns
to the template attributes
22 May 2017
Metadata Delivery in a Fact Build
Provide a name and define the
module properties for each
metadata delivery
22 May 2017
Metadata Delivery Modules
To deliver BI metadata, you must declare dimension
delivery modules for each dimension, but you do not need
to deliver the dimension data.
Select a fact table and columns
from the table that you want to
deliver
Disable dimension deliveries if you do not
want dimension data to be delivered through
the fact build
Specify a source and a dimension delivery
for each dimension that you want to deliver
Dimension deliveries
22 May 2017
Partitioned Delivery
Using DecisionStream, you can set vertical partitioning, horizontal partitioning, or both.
Vertical Partitioning
Horizontal Partitioning
Horizontal + Vertical Partitioning
22 May 2017
Vertical Partitioning
Vertical partitioning is performed in a fact delivery by subscribing to the elements to be
delivered.
Columns included
in Fact Delivery
Columns excluded
from Fact Delivery
22 May 2017
Module 5
Dimension Builds
22 May 2017
What is a Dimension Build?
A dimension build delivers data that describes a single dimension to the data mart.
This may take the form of one or several tables.
Levels of the
hierarchy
Reference
structure to
be delivered
Dimension
Build
Dimension
table
created
22 May 2017
Target
Data Mart
Dimension Builds
A dimension build can deliver dimension data for many fact builds. Fact builds can
also be used to deliver dimension data,but this may mean several fact builds
delivering the same data,resulting in a duplication of effort. Dimension builds deliver
only dimension data.
The main use for dimension builds is to deliver conformed dimensions that form the
dimensional reference for coordinated Data marts.
Each dimension build acquires data from a reference structure
that you specify and delivers this, as dimension data, to the
target data mart.
22 May 2017
Dimension Builds
Set Up Reference Structures
For each reference dimension that you have set up, you must set up a hierarchy and
the required templates. Each hierarchy models a view of the associated business
dimension.
DecisionStream supports hierarchies based on
• the columns of one table (star schema)
• multiple tables (snowflake schema)
• the rows of one table (parent-child schema)
• parent-child structures without named levels (auto-level hierarchies)
22 May 2017
Dimension Builds
Creating the Product Hierarchy
You create the Product hierarchy from
related tables in the GO_Sales database.
The hierarchy follows a one-to-many
relationship between tables.
A product line can have many product
types, but a product type can have
Only one product line. Each product
type can have many products, but a
product can have only one product
type.
22 May 2017
Dimension Builds
Steps to Select the Hierarchy Structure
1. From the Tools menu, click
Hierarchy Wizard.
2. Click Create the hierarchy from
multiple tables (Snowflake Schema),
and then click Next.
Steps to Define the Hierarchy
1. In the Enter the name of the
hierarchy box, type Product
2. In the Select the reference
dimension to use for this hierarchy
box, click Product, and then click
Next.
22 May 2017
Dimension Builds
Steps to Define the All Level
DecisionStream provides the option of creating a
static All level at the top of the hierarchy.
• Click Next to accept the defaults and create the All
level.
Steps to Define the ProductLine Level
1. Click the ALLProduct level of the Product
hierarchy, and then click Add.
The Level Details window appears.
2. In the Name box, type ProductLine
3. In the Source database box, click GO_Sales.
4. In the Source table box, click Browse for table, and
then double-click the ProductLine table.
5. Move each table column from the Available
attributes pane to the Chosen Attributes pane.
6. For the ProductLineCode attribute, select the Id
check box.
7. For the ProductLine attribute, select the Caption
check box and Click OK.
22 May 2017
Dimension Builds
Steps to Define the ProductType Level
1. Click the ProductLine level of the Product hierarchy, and then click Add.
2. In the Name box, type ProductType and In the Source database box, click GO_Sales.
4. In the Source table box, click Browse for table, and then double-click the ProductType table.
5. Move each table column from the Available attributes pane to the Chosen attributes pane.
6. For the ProductTypeCode attribute, select the Id check box.
7. For the ProductType attribute, select the Caption check box.
8. For the ProductLineCode attribute, select the Parent check box. When you execute a build,
DecisionStream creates links between a level and its parent by matching the Parent attribute of the
current level with the ID attribute of the level above. Then Click OK.
Steps to Define the Product Level
1. Click the ProductType level of the Product hierarchy, and then click Add.
2. In the Name box, type Product and In the Source database box, click GO_Sales.
4. In the Source table box, click Browse for table, and then double-click the Product table.
5. Move the ProductNumber, ProductName, and ProductTypeCode table columns from the
Available attributes pane to the Chosen attributes pane.
6. Select the Id check box for ProductNumber and Caption check box for ProductName
attribute.
7. For the ProductTypeCode attribute, select the Parent check box and Click OK.
22 May 2017
Dimension Builds
Steps to View the Product Hierarchy
Visualization
• In the Tree pane, click the Product
hierarchy.
The Visualization pane should look Like this:
The next task is to create the Product
dimension build and use this to
Deliver The Product hierarchy.
22 May 2017
The Dimension Build Wizard
The Dimension Build Wizard guides you step-by-step through
the process of creating a new dimension build.
22 May 2017
Dimension Builds
Steps to Create a Dimension Build
1. From the Tools menu, click Dimension Build
Wizard. The Dimension Build wizard appears.
2. In the Name box, type Product and then click Next.
3. In the Schema Type box, click Star Schema.
5. In the Dimension to be delivered box, click Product.
6. In the Reference item to be delivered box, click
Product (H).
7. In the Deliver into connection box, click DS_Marts,
and then click Next.
8. Click Next to accept default naming conventions.
9. Click Next to accept the default build features.
10. Click Next to accept the default build properties.
When using conformed dimensions in a data mart, it
is a good idea to use surrogate keys rather than
business keys to reference the dimensions.
11. Select the Add surrogate keys to the
dimension tables check box, and then click Next.
12. Click Finish to close the Dimension Build wizard.
22 May 2017
Dimension Builds
Steps to View the Dimension Build
• In the Tree pane, click the Product dimension build.
The Visualization pane should look like this:
Note that DecisionStream has created a template,
D_Product, which is used the deliver the primary
reference structure.
Steps to Deliver the Dimension Build
1. Right-click the Product dimension build, and
then click Execute.
The Execute dialog box appears.
2. In the Execution mode box, click Normal.
3. Click OK.
DecisionStream executes the dimension build in a
command window.
4. When the dimension build execution is complete,
press Enter to close the commandwindow.
22 May 2017
Dimension Build Schemas
• One dimension table:
– Star schema
– Parent-child schema
• Several dimension tables:
– Snowflake schema
– Optimal star schema
– Optimal snowflake schema
22 May 2017
Star Schema
Sales Staff
• Sales Rep Cd
• Sales Rep Name
• Sales Area Cd
• Sales Area Desc
Customer
• Customer Cd
• Customer Name
• Customer Type Cd
• Customer Type Desc
Order Fact
• Customer Cd
• Sales Rep Cd
• Product Cd
• Order Date
• Order_Qty
• Order_Line_Value
Date
• Order Date
• Week
• Month
• Year
Customer
Type
Sales
Area
Product
Line
Customer
Sales
Rep
Product
Type
Order
Header
Product
Order
Line
Product
• Product Cd
• Product Name
• Product Type Cd
• Product Type Desc
• Product Line Cd
• Product Line Name
22 May 2017
Snowflake Schema
A time-based dimension is
typically not snowflaked
Customer Type
• Customer Type Cd
• Customer Type Desc
Customer
• Customer Cd
• Customer Name
• Customer Type Cd
Date
• Order Date
• Week
• Month
• Year
Order Fact
• Customer Cd
• Sales Rep Cd
• Product Cd
• Order Date
• Order_Qty
• Order_Line_Value
Sales Rep
• Sales Rep Cd
• Sales Rep Name
• Sales Area Cd
Product Line
• Product Line Cd
• Product Line
Name
Product Type
• Product Type Cd
• Product Type Desc
• Product_Line Cd
Product
• Product Cd
• Product Name
• Product Type Cd
Sales Area
• Sales Area Cd
• Sales Area Desc
22 May 2017
Parent-Child Schema
Customer
• Customer CdCustomer
•
• Customer Name
Name
• Customer
• Customer Parent
Cd
• Customer Parent
• a parentNot
•
child
relationship
•
•
Order
OrderFact
Fact
• Cd
••Customer
••Sales Rep Cd
••Product Cd
••Order Date
Order Qty
••Order_Qty
Order Line Value
••Order_Line_Value
Organization
•
Sales Staff • Sales Rep Name
• Sales Rep Cd
• Sales Rep Parent
• Sales Rep Name
• Sales Rep Parent Cd
Date
• Order Date
• Week
• Month
• Year
Product
•
• Product Name
• Product Parent
Product
• Product Cd
• Product Name
• Product Parent Cd
22 May 2017
Dimension Builds
The main tasks required to define and use a Conformed dimension are
. Create a primary reference structure.
. Deliver the primary reference structure using a dimension build.
. Create a secondary reference structure using the dimension data delivered for the
primary reference structure.
. Associate the secondary reference structure with a fact build.
22 May 2017
Module 6
Job Streams
22 May 2017
Job Control and the Data Warehouse
The basic processes of the data warehouse require automation for the warehouse to
function effectively.
Raw Data
Target Data Mart
Automation of:
• Data Extraction
• Data Transformation
• Data Loading
• Exception/Error Handling
• Logging/Notification
22 May 2017
What Is a JobStream?
• A JobStream is a DecisionStream component used to coordinate data mart management tasks.
• The JobStream Visualization pane helps to understand the JobStream workflow.
22 May 2017
Characteristics of JobStreams
• A JobStream is associated to a single catalog.
• Nodes are processed in series or in parallel.
• A JobStream is aware only of objects in its catalog but can call
system commands (for example, databuild.exe) that can reference
other catalogs.
Catalog-specific JobStreams
contained in separate folder
Nodes within a single JobStream
22 May 2017
Add a JobStream to a Catalog
JobStreams are added to a catalog and are specific to that catalog.
Enter a name and other
properties for the
JobStream
22 May 2017
Add Variables to a JobStream
Define variables to use when creating procedure, condition, and SQL nodes.
The COUNTER JobStream variable is
used in the Counter Test node
22 May 2017
Main Components of a JobStream
Build
SQL
Procedure
Condition
Job
a fact or dimension build to be executed
SQL statement(s) to be executed
functions, variables, or actions to be performed
tests conditions for branching to other nodes
a container for one or more nodes that
can include nested jobs
22 May 2017
Build Node
Add a fact or dimension build node to a JobStream to automate the extract-transform-load
process.
Fact build that uses conformed
dimensions
Conformed dimension builds
22 May 2017
SQL Node
Use an SQL node to include SQL statement(s) to be executed during the job control process.
Database referenced by the SQL
SQL statements to create indexes
on a fact table previously loaded
by the JobStream
22 May 2017
Procedure Node
Use a Procedure Node to coordinate processing around a build.
The Execution Log procedure
writes a message to the log file
using the LogMsg function
22 May 2017
Result Variable
• All job nodes (except condition nodes)
set the Boolean variable RESULT to
indicate success or failure.
• Condition nodes check RESULT by
default.
• A node may deliver its result to another
variable if specified.
22 May 2017
Condition Node
Use a condition node to test a condition before executing another JobStream component.
If the condition specified in
Counter Test is False, then End
Automation Report executes
22 May 2017
Job Node
Use a job node to break large jobs into a series of smaller jobs.
If the FactBuild job node is
reached, a separate process will
run that contains its own logic
22 May 2017
Link JobStream Nodes
Each node in a JobStream process must be linked to at least one other node.
The Drop Indexes SQL node will be
completed BEFORE the SalesFact
build node that it is linked to
22 May 2017
Execute Nodes Sequentially or In Parallel
These nodes are run in parallel
These nodes are run sequentially
22 May 2017
Action on Failure
You can specify what DecisionStream does when a node fails.
CONTINUE: Continue processing the
next node
TERMINATE: Terminate the current
flow--continue with the next flow
ABORT: Stop ALL processing
immediately
22 May 2017
Run a JobStream
22 May 2017
Module 7
SQL Text Designer
22 May 2017
SQLTXT Designer
You use the SQLTXT Designer to configure the
SQLTXT definition files used within
DecisionStream to access text files as though
they are in an SQL database.
SQLTXT Definition Files
A SQLTXT definition file (.def) defines a virtual
database of text file tables. It associates each
Table with a text file in the database directory. It
also specifies the structure of the tables.
Steps to create a SQLTXT definition file
1. In the SQLTXT Designer, from the File menu, click New.
2. From the File menu, click Save As.
3. In the File name box, type a name for the file.
Note: The definition file must have the extension .def.
4. Click Save.
22 May 2017
SQLTXT Designer
Steps Import Delimited Records with Delimited
Columns
1. Open the appropriate SQLTXT database.
2. From the Edit menu, click Import Table Data.
The Import SQLTXT Table Data File dialog box appears.
3. Select the text file to import, and click Open.
4. Click Delimited records.
5. In the Record delimiter box, click the delimiter
used to separate individual records in the file.
6. In the Number of header rows box, enter the
number of header rows in the file. The SQLTXT
Designer uses these header rows to name the
columns.
7. In the Decimal symbol box, type a single
character that is used for the decimal point in
numeric values.
22 May 2017
SQLTXT Designer
8. In the Character set box, enter the encoding
used in the text file.
9. In the Digit grouping symbol box, type the
character that is used for the thousands
separator in numeric values.
10. In the Preview rows box, enter the number of
rows that you want to view in the wizard.
11. Click Next.
12. Select Delimited columns.
13. In the Delimiter type box, click the delimiter
used to separate the columns in the table.
14. In the Quote character box, click the quotation
character used to enclose individual columns
within a record.
15. Click Next.
22 May 2017
SQLTXT Designer
DecisionStream automatically names and
attempts to specify the data type of each column.
You can override these selections.
16. To rename a column, click the column name
and type a new name.
17. To change a data type, double-click the
appropriate column, and in the Data type dialog
box, click a different data type. If you select
NUMBER, you must also specify the number of
digits expected after the decimal point in the
Scale box. If you select DATE, you must also
specify the format of the date in the Format box.
18. Click OK to close the Data type dialog box.
19. Click Process All Records to ensure that all
record lengths and data types are correct.
20. Click Finish. The wizard adds the table to the
database, and the SQLTXT Designer appears.
22 May 2017
SQLTXT Designer
Steps to define a table manually in SQLTEXT
Designer:
1. Click the Database icon.
2. From the Edit menu, click Insert Table. An empty
table appears in the tree.
3. Click the new table and, from the Edit menu, click
Rename.
4. Type the new name for the table.
5. In the Header lines text box, type the number of
header rows present in the file.
6. In the Trailer lines text box, type the number of
trailer lines in the file.
SQLTXT ignores these rows when it accesses data
in the file.
7. In the Character set box, enter the encoding used in
the text file.
You can click an encoding type from the list, or type
the code page into the box.
22 May 2017
SQLTXT Designer
8. In the Action on row error box, click the action to apply when faulty data is read.
9. Select or clear the Remove trailing whitespace from fields check box as appropriate.
10. Specify whether columns in the input text file are of fixed length or delimited .
11. In the Record delimiter box, click the delimiter used to separate individual records.
12. If you selected Delimited records, in the Column delimiter box, specify the delimiter
used to separate the columns in the table.
13. If you selected Delimited records, click the quotation character used to enclose
individual columns within a record.
14. If you selected Delimited records, select the Ignore extra columns check box to ignore
any excess fields.
22 May 2017
SQLTXT Designer
Adding Columns to the table manually:
1. Click the table to which you want to add a
column.
2. From the Edit menu, click Insert Column.
A new column appears beneath the selected
table.
3. In the Data type box, click the data type for the
column, and specify appropriate length for that
column.
4. If the column can accept null values, select the
Nullable check box.
5. If the column forms part of the primary key of the
table, select the Key check box.
22 May 2017
SQL Term
Creating an SQL Query (SELECT Statement)
1. From the Tools menu, click SQLTerm.
2. In the box at the top of the SQLTerm window,
click the database with which you want to work.
3. Use the Cognos SQL check box to specify
whether you want to use the native SQL or
Cognos SQL for the database you are accessing
when you construct the SQL statement.
Select or clear the check box as appropriate.
Tip: Using Cognos SQL makes it easier to port your
project between database platforms.
Note: The default for the Cognos SQL check box
is determined by whether you selected the
Cognos SQL check box in the Connection
Properties dialog box.
22 May 2017
SQL Term
4. In the Database Objects pane, expand the tree,
and then click the table or column for which
you want to add a SELECT statement.
5. To generate the SQL statement use one of the
following methods:
• Right-click, and then click either Add Table
Select Statement, or Add Column Select
Statement.
• Click the Source SQL tab. Hold down Ctrl and
drag the table or column you selected in
the Database Objects pane to the Source SQL tab.
DecisionStream clears any existing query that
may be on the Source SQL tab and generates the
SQL SELECT statement for the selected table or
column.
6. You are now ready to execute the SQL
statement.
22 May 2017
Module 8
Command Line interface
22 May 2017
Command Line Interface
Executing a Fact build from a DecisionStream Catalog
DATABUILD -c<catdb_driver> <catdb_items> <databuild_name>[<options>]
Where
<catdb_driver> The database driver that DecisionStream must use to connect to the
catalog. <catdb_items> The connection string that DecisionStream must pass to the
database driver to connect to the catalog. <databuild_name> The name of the build that
you want to execute. When you invoke DATABUILD with this syntax, it executes the named
build that resides in the named catalog.
Example
DATABUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" US_Sales
This command invokes DATABUILD to execute the build named US_Sales. To retrieve the
Build definition, DATABUILD connects to the ODBC data source named 'SalesCat' as user
Sales', with password 'ds123'.
22 May 2017
Command Line Interface
To Execute a Dimension Build from a Catalog
DIMBUILD -c<catdb_driver> <catdb_items> <dimbuild> [<options>]
Where
<catdb_driver> The database driver that DecisionStream must use to connect to the
catalog.
<catdb_items> The connection string that DecisionStream must pass to the database
driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS',
to a catalog that resides in an ODBC data source named “Sample”, you would specify:
"DSN=Sample;UID=DBA;PWD=PASS"
<dimbuild> The name of the dimension build that you want to execute.
Example
DIMBUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" Period
The command above invokes DIMBUILD to execute the dimension build named Period. To
retrieve the dimension build definition, DIMBUILD connects to the ODBC data source named
'SalesCat' as user 'Sales', with password 'ds123'.
22 May 2017
Command Line Interface
RUNDSJOB
This command executes a JobStream from a DecisionStream catalog.
Syntax
RUNDSJOB <catdb_driver> <catdb_items> <jobstream> [<options>]
<options> ::= [-e<runenv>] [-j<subjob>] [-A<dbalias_file>] [-C]
[-L<logfile>] [-P] [-O] [-R] {-V<name>[=<value>]}
Where
<catdb_driver> The database driver that DecisionStream must use to
connect to the catalog that contains the JobStream.
<catdb_items> The connection string that DecisionStream must pass
to the database driver to connect to the catalog. For example, to connect as user 'DBA', with
Password 'PASS', to a catalog that resides in an ODBC data source named “Sample”, you
would specify:"DSN=Sample;UID=DBA;PWD=PASS"
<jobstream> The name of the JobStream that DecisionStream
must execute.
-e Denotes the run environment for the JobStream.
22 May 2017
Command Line Interface
RUNDSJOB – Continue …
<runenv> The run environment that DecisionStream must apply to identify the correct
database alias.
-j Denotes the subjob of the JobStream that DecisionStream should execute.
<subjob> JobStreams can nest jobs. This option enables you to execute a job that is nested
within the JobStream.
-A Denotes the text file that contains the database alias definitions that DecisionStream
should use for the JobStream.
<dbalias_file> The name and path of the text file that contains database alias
definitions that DecisionStream must use for the build.
-C Executes the JobStream in check-only mode. That is, DecisionStream executes the
JobStream but does not deliver any data.
-L Causes DecisionStream to log JobStream progress to the specified file.
<logfile> The name and path of a text file to which DecisionStream should log build
progress.
-P Causes DecisionStream to pause at the end of JobStream execution and to prompt for a
keystroke before terminating.
22 May 2017
Command Line Interface
RUNDSJOB – Continue …
-O Causes DecisionStream to execute any builds in the JobStream in object creation mode.
That is, DecisionStream executes the builds and creates any required fact and dimension
tables and metadata structures, but does not deliver data to them.
-R Causes DecisionStream to restart the JobStream at the point of a previous interruption the
JobStream restarts at the earliest node that did not successfully complete.
-V Denotes that the parameter is a named variable and value pair.
<name> The name of a variable that DecisionStream must use during item substitution.
<value> The value that the corresponding named variable takes.
Example
RUNDSJOB ODBC "DSN=Sales;UID=admin" example_job -jbuild_fiscal -R
This example invokes RUNDSJOB to restart the subjob named 'build_fiscal' that is nested in
The job named 'example_job'. The DecisionStream catalog resides in the ODBC data source
Named 'Sales', to which DecisionStream connects as user 'admin' with no password.
22 May 2017
Command Line Interface
CATEXP
This command exports a catalog item to a DecisionStream file-based specification.
Syntax
CATEXP <catdb_driver> <catdb_items> <export_file> <comp_type>
[<comp_name>] [-a] {-V<name>[=<value>]} <comp_type> ::=
A|B|C|D|F|H|J|L|S|T
Where
<catdb_driver> The database driver that DecisionStream must use to connect to the
catalog.
<catdb_items> The connection string that DecisionStream must pass to the database
driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS',
to a catalog that resides in an ODBC data source named 'Sample', you would specify:
"DSN=Sample;UID=DBA;PWD=PASS“
<export_file> The path and name of the file
to which you want to export the catalog component.
<comp_type> The type of component that you want to export.
22 May 2017
Command Line Interface
CATEXP – Continue …
-a Causes DecisionStream to append the exported component to the named file.
<name> The name of a variable.
<value> The value that the associated variable takes.
<comp_type> The type of component that you want to export. The comp_type
values have the following meanings:
22 May 2017
Command Line Interface
CATIMP
This command imports a text-based component definition into a DecisionStream catalog.
Syntax
CATIMP <catdb_driver> <catdb_items> <import_filename>
<filetype> [<comp_name> [<caption>[<notes>]]] [-d<dbalias>][-o]
[-I] {-V<name>[=<value>]} <filetype> ::= B|C|D|F|J|R|S
Where
<catdb_driver> The database driver that DecisionStream must
use to connect to the catalog.
<catdb_items> The connection string that DecisionStream must pass to the database
driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS',
to a catalog that resides in an ODBC data source named 'Sample', you would specify:
"DSN=Sample;UID=DBA;PWD=PASS“
<import_filename> The path and name of the text file that defines the component.
<filetype> The type of component that you want to export.
22 May 2017
Command Line Interface
CATIMP – Continue…
<comp_name> The name of the individual component that you want to import.
This is optional for reference structures, aliases, and functions, but mandatory
for other component types.
<caption> An optional caption, or business name, that DecisionStream should
use for the item for display or presentation purposes.
<notes> Optional notes that appear within DecisionStream Designer.
22 May 2017
Command Line Interface
CATIMP – Continue…
Example
CATIMP ODBC "DSN=SalesCat" SalesBld.bld B US_Sales "US Sales -o
This command invokes CATIMP to import the build US_Sales from the text file
Sales.bld, into the catalog that resides within the ODBC data source named
'SalesCat', overwriting any build with the same name.
Within the catalog, CATIMP gives the build US_Sales the caption 'US Sales'.
CATIMP connects to the named data source as the default user, with no password.
22 May 2017
Command Line Interface
CATIMP – Continue…
Order of Import
Because DecisionStream validates objects that reside in a catalog, you must import text
Definitions in dependency order. For example, do not import a build until the catalog contains
the collections and all the aliases and hierarchies that it uses. Normally, you should import
text definitions in this order:
1. Aliases
2. User-defined functions
3. Dimension table templates
4. Reference structures (hierarchies and lookups)
5. Collections
6. Builds and dimension builds
7. JobStreams
22 May 2017
Definitions
• Build
This is a DecisionStream Unit of work that goes through the steps of
Acquisition, Transformation, and Delivery. There are two types of
build, fact builds and dimension builds.
• Collection
When DecisionStream is driven from a text file specification, the
transformations are split into a build and a collection. The build deals
with the delivery of data and the collection deals with the sources of
data and the transformation process.
• Component
A fact build, dimension build, collection, hierarchy, lookup jobstream,
user-defined function, template or connection in a catalog.
22 May 2017
Module 9
Definations
22 May 2017
Definitions
• Condition Node
A condition node provides a branching mechanism between nodes for
conditional execution. Each condition node can have many nodes linking to it,
but only two output links (True, and False).
• Connection
All the information required to make a connection to a database. The actual
information required varies according to DBMS and connection method.
• Data Mart
A database that holds data for analysis. It usually implies that the data and its
organization is of limited scope or is targeted at a particular group of users and
business subjects, for example orders.
• Data Source
Somewhere from which the DecisionStream can acquire data.
22 May 2017
Definitions
• DataStream
A DecisionStream object that gathers together all the data sources
from which the build or reference structure acquired data. Data
streams merge these sources to present a consider whole to other
DecisionStream objects.
• Designer
DecisionStream Designer is the Windows software that you use to
produce build specifications and Dimensional Frameworks.
• Dimension
A Key Component of a business by which you want to analyze your
data. For example, the products that your company sells and the
retail outlets in which these products are sold.
• Dimension Build
A Decision Stream build process that delivers only dimension data.
22 May 2017
Definitions
•
Dimension Template
A dimension template lists the columns that may exists in a table of dimension
data, together with the behaviour of each column.
•
Domain Type
It is the domain that is used to deliver dimension data at the end of a build. The
two types are Dynamic ( where the domain is build up from data sources as
rows are processed) and Reference (where the reference object associated
with the dimension is taken as the domain)
•
Factbuild
A DecisionStream build that delivers fact data. May optionally deliver
dimension data, and metadata, allowing a private datamart to be built from
within a single build.
•
Foster Parent
An artificially introduced member that acts as a parent for members that either
have no defined parent or whose defined parent cannot be found at the next
hierarchy level.
22 May 2017
Definitions
• JobStream
A DecisionDtream object that combines other DecisionStream
object to form a logical process.
• Job Node
A job node is actually a JobStream within a JobStream. This
allows larger jobs to be broken down into a series of smaller jobs.
When DecisionStream encounters a job node it processes all the
steps within the sub job, and only when these steps are complete,
does it move on to the next node in the sequence.
• Leaf Node
In a hierarchy, a member that has no children.
• Lookup
A simple reference object that consists of members that are not
arranged into hierarchical levels.
22 May 2017
Definitions
• Mapping
The relationship between columns (from an SQL query) and the
DecisionStream objects to which they relate. DecisionStream has mappings
that relate data sources to transformation model elements and structure data
columns to level attributes.
• Message Frequency
The rate at which DecisionStream generates messages as it executes a build.
Input message frequency refers to how many data rows DecisionStream
acquires between messages. Output message frequency refers to how many
data rows DecisionStream delivers between messages.
• Metadata
Data that describe the delivered data. Typically, metadata defines the data
structure and the domain of each dimension. It can also provide aliases that
reporting tools can use to provide meaningful descriptions.
22 May 2017
Definitions
• Never Output
An indication on a transformation model element that DecisionStream must
not make the element available for delivery.
• Object Creation Mode
A build execution mode in which fact tables, dimension tables, and metadata
are created but no fact or dimension data is delivered.
• Output Filter
A filter that can be specified in a delivery module to restrict delivery of rows
to the target system. The output filter takes any expression that evaluates to
TRUE or FALSE.
• Output Level
A level of hierarchy at which DecisionStream will output data. Output data
may be generated by aggregation.
22 May 2017
Definitions
• Pivot
To move a dimension from on axis to another. For example, consider a
relational table that has a single column (called product) that holds all product
values. Assume that this has one column fro each business measure (units,
revenue, etc.) The product column could be pivoted to produce one column for
each product and the business measures pivoted to produce one row for each
measure.
• Procedure Node
A procedure node can contain one or more DecisionStream functions or
statements. This node can be used to coordinate processing around builds for
such activities as checking for input files, sending mail messages and alerts,
and generating custom logging and auditing messages.
• Ragged Hierarchy
A hierarchy that has leaf nodes at more than one level. Also called an
unbalanced hierarchy.
22 May 2017
Definitions
• Reference Explorer
A DecisionStream object that gathers together all reference structures that relate
to a particular business dimension.
• Reject File
A text file in which DecisionStream stores records that the engine rejects. In
these files, each rejected record has the reason for rejection as its first field.
• Sample Rate
When reading from a data source, DecisionStream can apply a sample rate so
that it does not process every row. For example, a sample rate of 10 results in
DecisionStream processing every tenth row and ignoring the intermediate rows.
• Slowly Changing Dimension
A business dimension in which one or more non-key values may change
without a corresponding change in the key values.
22 May 2017
Definitions
• SQL Node
Use an SQL node to enter a single SQL statement that DecisionStream
performs when the SQL node is reached in the JobStream flow.
DecisionStream automatically commits the successful SQL statements, but roll
back if the statement fails.
• SQLTerm
A terminal monitor application that provides a common interface to the
various relational database products that DecisionStream supports.
• SQLTXT Designer
It is a TOOL which enables text files to be described in a way that
subsequently allows SQLTerm to access the data that the text files hold.
• Transformation
The re-expression of a stream of data in some different format. Typically the
merging and aggregation of source data to provide delivery elements.
22 May 2017
Definitions
• Star Schema
A star schema represents the dimension in a single table. Each level of the
delivered hierarchy is represented by one or more columns of this table. In
order to use the Dimension Build wizard to create a dimension build that
supports slowly changing dimensions, you must use this schema type.
• Surrogate Key
Surrogate keys are numeric keys that can replace business keys in the
context of a data mart. Whereas the natural ID may be text-based, the
surrogate key is always an integer.
• Tree Pane
The left pane of the Designer window where DecisionStream arranges the
objects of the catalog in a tree-like hierarchical fashion.
• Transformation Model
Collection of dimension, measure, attribute and derivation elements in a fact
build.
22 May 2017
Thank You
22 May 2017