Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Microsoft Jet Database Engine wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Clusterpoint wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Functional Database Model wikipedia , lookup
COGNOS Decision Stream Cognos CoE Internal and Confidential Agenda • Getting Started • Components of DecisionStream • Definitions • DecisionStream Catalog • Database Connections • Fact Builds • Dimension Builds • JobStreams • SQLTXT Designer • SQL Term • Command Line Interface 22 May 2017 Modules 1. Getting Started 2. Create a Catalog 3. Fact builds 4. Data Delivery Modules 5. Dimension Builds 6. Job streams 7. SQL Text Designer 8. Command Line Interface 9. Definitions 22 May 2017 Module 1 Getting Started 22 May 2017 What is DecisionStream? • The primary purpose of DecisionStream is to create integrated data marts by: – extracting operational data from multiple sources – merging and transforming the data to facilitate reporting and analysis – delivering transformed data into coordinated data marts 22 May 2017 The Purpose of DecisionStream Transformations Data Data OLTP OLTP OLTP OLTP Data Integrated + x/y - Data Marts OLTP 22 May 2017 Decision Stream Architecture Platforms Windows NT,95,98, 2000 Desktop Design Client API Scripts DecisionStream Process Metadata NT or UNIX Server HPUX AIX Solaris IBM AIX 4.2 COMPAQ TRU64 Operational Sources DB2 Oracle SQL Server Informix ODBC Sybase Text MetaData Targets Impromptu Transformer Architect Custom DBMS Flat Files Microsoft SSAS Essbase ERP Server Engine 22 May 2017 How DecisionStream Creates Data Marts Dim Data Sources Dimensional Framework Product Time Location 1 Sales 3 5b. Dimension Delivery Data Source 5a. 2 Fact Delivery 5c. Data Source MetaData Delivery 4 22 May 2017 DecisionStream Interface 22 May 2017 Interface Objects • The core elements in a DecisionStream catalog are: – Builds – DataStreams – Transformation Models – Delivery Modules 22 May 2017 Interface Objects (Cont’d) • The core elements in a DecisionStream catalog are: – JobStreams – Dimensions • Hierarchies and Lookups • Templates – Connections – Functions 22 May 2017 DecisionStream Tools 22 May 2017 Module 2 Create a Catalog 22 May 2017 What is a Catalog? A catalog is a container for DecisionStream metadata. 22 May 2017 DecisionStream Catalog • A DecisionStream Catalog provides a central repository for the information that defines how DecisionStream extracts, transforms, and delivers data. • You can store a DecisionStream Catalog in either a dedicated database or a database that you use for other purposes, such as a target data mart. • A Catalog consists of a set of data tables that can reside in any database for which DecisionStream has create permissions. You can create only one Catalog per database. 22 May 2017 DecisionStream Catalog Diagram illustrates • How DecisionStream extracts and transforms data from the data sources and how it delivers data to the target marts • The flow of data from the data sources to the target data marts 22 May 2017 Shared Items in a Catalog • Library items can be used throughout the catalog in a variety of different builds and JobStreams. • A catalog library consists of: – Dimensions – Connections – Functions 22 May 2017 DecisionStream Catalog Create a Catalog You can create a Catalog in the DS_New sample database. Steps 1. From the File menu, click New Catalog. If you are prompted to disconnect from the current Catalog, click OK. The New Catalog dialog box appears. 2. In the Name box, type MyCatalog and then click Next. 3. In the left pane, click ODBC. 4. In the Data Source Name box, click DS_New. 5. Click Finish. DecisionStream creates the Catalog tables and opens the new Catalog. 6. From the File menu, click Save Catalog. 22 May 2017 Backup and Restoration of a Catalog • Backup files are useful for recovery and emergency situations. • Write the catalog tables to an ASCII text file for later recovery. 22 May 2017 Database Connections Creating an ODBC Connection Steps to define connections to ODBC databases set up for the Great Outdoors Company. Steps 1. Open the MyCatalog catalog that you created in Chapter 2. 2. In the Library, right-click Connections, then click Insert Connection. 3. In the Alias box, type GO_Sales 4. Click the Connection Details tab. 5. In the left pane, click ODBC. 6. In the Data Source Name box, click GOScer3. 7. Click OK to close the Connection Properties dialog box. 22 May 2017 Database Connections Creating a connection to SQLTXT Database Steps 1. In the Library, click the Connections folder. 2. From the Insert menu, click Library, and then click Connection. 3. In the Alias box, type DS_Sources 4. Click the Connection Details tab. 5. In the left pane, click SQLTXT. 6. In the Definition File box, click Browse, then select the DS_Sources.def file in the Cognos samples directory. 8. Click OK to close the Connection Properties dialog box. 22 May 2017 Database Connections Creating a connection to Oracle Database Steps 1. In the Library, click the Connections folder. 2. From the Insert menu, click Library, and then click Connection. 3. In the Alias box, type DS_OracleSrc 4. Click the Connection Details tab. 5. In the left pane, click Oracle SQL*Net. 6. Enter the values for User Name, Password and SQL*Net Service. 8. Click OK to close the Connection Properties dialog box. 22 May 2017 Module 3 Fact Builds 22 May 2017 Fact Builds A DecisionStream fact build specifies how to extract, transform, and deliver data. Data may come from multiple data sources, Each of which has its own naming and storage conventions. A fact build can • acquire data from one or more data sources • merge and clean the source data • aggregate the data with reference to the dimensional framework • deliver the fact and dimension data to one or more data marts • deliver metadata to OLAP systems and Cognos BI applications 22 May 2017 The Fact Build Process Structure Data Sources Dimensional Framework Deliver 10100101 … Transaction Data Sources Acquire Star Join Merge Transform Aggregate Partition Filter 22 May 2017 Fact Build Data Elements Dimension A dimension key or data that is used to qualify a measure. Measure Attribute Quantitative data or values that can be aggregated. These are usually Key Performance Indicators (KPIs). Additional information that is not a dimension element or measure but that may be of interest. Derivation A calculation or transformation. 22 May 2017 Fact Build Wizard The Fact Build Wizard guides you step-by-step through the process of creating a new fact build. 22 May 2017 Types of Fact Builds Fact build types provide default settings for the delivery of data marts. 22 May 2017 The Fact Build Wizard: Add a Data Source A data source for the fact build is chosen from the available connections. This source contains the transactional data to be transformed. 22 May 2017 The Fact Build Wizard: Create the SQL Data Source Tables and columns to be read from SQL statement created 22 May 2017 The Fact Build Wizard: Map to Fact Build Elements Columns returned from source data by SQL Elements of the Transformation Model 22 May 2017 The Fact Build Wizard: Modify Attributes The elements of the Transformation Model are created automatically by the Fact Build Wizard but may need to be modified. Click here to modify the types of elements Elements of the Transformation Model 22 May 2017 The Fact Build Wizard: Link to the Dimensional Framework Dimension elements are related to reference structures. 22 May 2017 The Fact Build Wizard: Fact Data Delivery 22 May 2017 Fact Build Visualization Dimensions associated with the build Transactional data sources Fact and Dimension tables Metadata delivery to Impromptu, PowerPlay, and Architect 22 May 2017 DataStream Mapping Visualization You can visualize the mapping of the SQL columns to the build elements by clicking the DataStream tab. 22 May 2017 Transformation Model Visualization Use the Transformation Model tab to view the linkage of the dimension elements to the Dimensional Framework. Mapped Reference Structures 22 May 2017 Fact Delivery Visualization By clicking the Fact Delivery tab, you can view the mapping of the elements to the target fact table. 22 May 2017 Delivery Modules • The three delivery module types are: – fact – dimension – metadata 22 May 2017 Fact Builds Steps to Execute the Fact Build 1. Right-click the BIMart fact build, and then click Execute. The Execute Build dialog box appears. 2. In the Execution mode box, click Normal, and then click OK. DecisionStream executes the Fact build in a command window, writing progress messages to the screen and execution log. 3. When the fact build execution Is complete, press Enter to close The command window. 22 May 2017 Run a Fact Build 22 May 2017 Create a Fact Build Manually 1 Add a build 3 Add and define the build elements 2 Define the source 4 Map the elements to the columns SQL columns Build elements 22 May 2017 Add a Dimension Element to the Fact Build Insert Associate and Set Select to roll up to the higher level(s) Select to include all records to avoid rejected data You have the option of adding the dimension element to deliveries after the fact table has been created Select to use the surrogate, if one is available, as a foreign key 22 May 2017 Add a Measure to a Build Insert and name it Specify a merge behavior Specify an aggregation method 22 May 2017 What is a Derivation? • Derivation is an element that DecisionStream calculates rather than obtains directly from the data source. • It derives from any build element. • It is an output-only element. ProductRevenue = Quantity * UnitPrice Quantity * UnitPrice 22 May 2017 When to Use a Derivation? • Use a derivation when: – a calculation needs to be stored in the data mart to be shared by users – the calculation is very complex and open to misinterpretation – certain key performance indicators (KPIs) must be derived – transformation is required 22 May 2017 Operators for Derivations Mathematical: – ( ) – + – – * – / Logical: =, != >=, <= >, <, <> IS AND, OR [NOT] LIKE [NOT] BETWEEN [NOT] IN NOT ( ……….. ) 22 May 2017 General Functions for Derivations Function Type Examples Text Char(), Concat(), Trim(), Length(),... Mathematical Abs(), Floor(), Trunc(), Round(), Mod() Logical If(), IfNull() Type Conversion ToChar(), ToDate(), ToInteger() Date AddToDate(), SysDate(), DaysBetween() Control RowsUpdated(), Exit(), SQL(), LogMsg(),.. Member Member(), Level(), IsAncestor() 22 May 2017 Control Statements Statements Assignment Return if… then… else… while… do… case… <block> Syntax <va ria ble> := <expression>; return <expression>; if<expression> then<block> else<block> while <expression> do <block> ca se <expression> of begin <va lue1>: <block> defa ult: <block> end begin<sta tement>;end 22 May 2017 How to Define a Derivation Insert Calculate Test 22 May 2017 When to Calculate a Derivation By default , a derivation is calculated after aggregation. You can choose to calculate the derivation before aggregation. Select the Calculate at Source box if you want to calculate first and then aggregate. Select an aggregation method AverageRevenue Quantity * UnitCost Revenue (Quantity*UnitCost) MEASURE MEASURE DERIVATION AVGRevenue AGGREGATION 22 May 2017 Module 4 Data Delivery Modules 22 May 2017 Delivery Process to a Single Data Mart Fact Build Delivery Modules Delivery Targets Dimension Delivery Fact Delivery Data Mart Metadata Delivery 22 May 2017 Fact Delivery Modules There are two groups of fact delivery modules. Table delivery modules Nontable delivery modules •Bulk Copy (BCP) •Text File •DB2 LOAD •DataStream •Informix LOAD •Essbase Direct Feed •ORACLE SQL *Loader •Essbase File Feed •Red Brick Loader (TMU) •Relational Table 22 May 2017 Fact Delivery: Relational Table Delivery Module Target database Target table Indexing options Refresh Type: • APPEND • TRUNCATE • REPLACE Elements subscription for delivery • UPDATE/INSERT • UPDATE Commit Interval: Key option 1 .. 100 commit … 200 commit … … 1000 22 May 2017 Fact Delivery: Text File Delivery Module Elements subscription for delivery Output Filename full directory path and file name or just file name. Line/Field Delimiter NL(new line) TAB, COMMA, SPACE, NONE(no delimiter), or a character string. 22 May 2017 Dimension Delivery in a Fact Build • In general you should not deliver dimensional data in a fact build. However, there are four exceptions to this rule: – a demo or proof of concept – data mart delivery from a warehouse – dimensional data available only in fact data – metadata delivery 22 May 2017 Dimension Delivery Module Type or select a name for dimension table Create or select a template for delivery Add Relational Table Add Dimension Table Template Attributes Map the data set columns to the template attributes 22 May 2017 Metadata Delivery in a Fact Build Provide a name and define the module properties for each metadata delivery 22 May 2017 Metadata Delivery Modules To deliver BI metadata, you must declare dimension delivery modules for each dimension, but you do not need to deliver the dimension data. Select a fact table and columns from the table that you want to deliver Disable dimension deliveries if you do not want dimension data to be delivered through the fact build Specify a source and a dimension delivery for each dimension that you want to deliver Dimension deliveries 22 May 2017 Partitioned Delivery Using DecisionStream, you can set vertical partitioning, horizontal partitioning, or both. Vertical Partitioning Horizontal Partitioning Horizontal + Vertical Partitioning 22 May 2017 Vertical Partitioning Vertical partitioning is performed in a fact delivery by subscribing to the elements to be delivered. Columns included in Fact Delivery Columns excluded from Fact Delivery 22 May 2017 Module 5 Dimension Builds 22 May 2017 What is a Dimension Build? A dimension build delivers data that describes a single dimension to the data mart. This may take the form of one or several tables. Levels of the hierarchy Reference structure to be delivered Dimension Build Dimension table created 22 May 2017 Target Data Mart Dimension Builds A dimension build can deliver dimension data for many fact builds. Fact builds can also be used to deliver dimension data,but this may mean several fact builds delivering the same data,resulting in a duplication of effort. Dimension builds deliver only dimension data. The main use for dimension builds is to deliver conformed dimensions that form the dimensional reference for coordinated Data marts. Each dimension build acquires data from a reference structure that you specify and delivers this, as dimension data, to the target data mart. 22 May 2017 Dimension Builds Set Up Reference Structures For each reference dimension that you have set up, you must set up a hierarchy and the required templates. Each hierarchy models a view of the associated business dimension. DecisionStream supports hierarchies based on • the columns of one table (star schema) • multiple tables (snowflake schema) • the rows of one table (parent-child schema) • parent-child structures without named levels (auto-level hierarchies) 22 May 2017 Dimension Builds Creating the Product Hierarchy You create the Product hierarchy from related tables in the GO_Sales database. The hierarchy follows a one-to-many relationship between tables. A product line can have many product types, but a product type can have Only one product line. Each product type can have many products, but a product can have only one product type. 22 May 2017 Dimension Builds Steps to Select the Hierarchy Structure 1. From the Tools menu, click Hierarchy Wizard. 2. Click Create the hierarchy from multiple tables (Snowflake Schema), and then click Next. Steps to Define the Hierarchy 1. In the Enter the name of the hierarchy box, type Product 2. In the Select the reference dimension to use for this hierarchy box, click Product, and then click Next. 22 May 2017 Dimension Builds Steps to Define the All Level DecisionStream provides the option of creating a static All level at the top of the hierarchy. • Click Next to accept the defaults and create the All level. Steps to Define the ProductLine Level 1. Click the ALLProduct level of the Product hierarchy, and then click Add. The Level Details window appears. 2. In the Name box, type ProductLine 3. In the Source database box, click GO_Sales. 4. In the Source table box, click Browse for table, and then double-click the ProductLine table. 5. Move each table column from the Available attributes pane to the Chosen Attributes pane. 6. For the ProductLineCode attribute, select the Id check box. 7. For the ProductLine attribute, select the Caption check box and Click OK. 22 May 2017 Dimension Builds Steps to Define the ProductType Level 1. Click the ProductLine level of the Product hierarchy, and then click Add. 2. In the Name box, type ProductType and In the Source database box, click GO_Sales. 4. In the Source table box, click Browse for table, and then double-click the ProductType table. 5. Move each table column from the Available attributes pane to the Chosen attributes pane. 6. For the ProductTypeCode attribute, select the Id check box. 7. For the ProductType attribute, select the Caption check box. 8. For the ProductLineCode attribute, select the Parent check box. When you execute a build, DecisionStream creates links between a level and its parent by matching the Parent attribute of the current level with the ID attribute of the level above. Then Click OK. Steps to Define the Product Level 1. Click the ProductType level of the Product hierarchy, and then click Add. 2. In the Name box, type Product and In the Source database box, click GO_Sales. 4. In the Source table box, click Browse for table, and then double-click the Product table. 5. Move the ProductNumber, ProductName, and ProductTypeCode table columns from the Available attributes pane to the Chosen attributes pane. 6. Select the Id check box for ProductNumber and Caption check box for ProductName attribute. 7. For the ProductTypeCode attribute, select the Parent check box and Click OK. 22 May 2017 Dimension Builds Steps to View the Product Hierarchy Visualization • In the Tree pane, click the Product hierarchy. The Visualization pane should look Like this: The next task is to create the Product dimension build and use this to Deliver The Product hierarchy. 22 May 2017 The Dimension Build Wizard The Dimension Build Wizard guides you step-by-step through the process of creating a new dimension build. 22 May 2017 Dimension Builds Steps to Create a Dimension Build 1. From the Tools menu, click Dimension Build Wizard. The Dimension Build wizard appears. 2. In the Name box, type Product and then click Next. 3. In the Schema Type box, click Star Schema. 5. In the Dimension to be delivered box, click Product. 6. In the Reference item to be delivered box, click Product (H). 7. In the Deliver into connection box, click DS_Marts, and then click Next. 8. Click Next to accept default naming conventions. 9. Click Next to accept the default build features. 10. Click Next to accept the default build properties. When using conformed dimensions in a data mart, it is a good idea to use surrogate keys rather than business keys to reference the dimensions. 11. Select the Add surrogate keys to the dimension tables check box, and then click Next. 12. Click Finish to close the Dimension Build wizard. 22 May 2017 Dimension Builds Steps to View the Dimension Build • In the Tree pane, click the Product dimension build. The Visualization pane should look like this: Note that DecisionStream has created a template, D_Product, which is used the deliver the primary reference structure. Steps to Deliver the Dimension Build 1. Right-click the Product dimension build, and then click Execute. The Execute dialog box appears. 2. In the Execution mode box, click Normal. 3. Click OK. DecisionStream executes the dimension build in a command window. 4. When the dimension build execution is complete, press Enter to close the commandwindow. 22 May 2017 Dimension Build Schemas • One dimension table: – Star schema – Parent-child schema • Several dimension tables: – Snowflake schema – Optimal star schema – Optimal snowflake schema 22 May 2017 Star Schema Sales Staff • Sales Rep Cd • Sales Rep Name • Sales Area Cd • Sales Area Desc Customer • Customer Cd • Customer Name • Customer Type Cd • Customer Type Desc Order Fact • Customer Cd • Sales Rep Cd • Product Cd • Order Date • Order_Qty • Order_Line_Value Date • Order Date • Week • Month • Year Customer Type Sales Area Product Line Customer Sales Rep Product Type Order Header Product Order Line Product • Product Cd • Product Name • Product Type Cd • Product Type Desc • Product Line Cd • Product Line Name 22 May 2017 Snowflake Schema A time-based dimension is typically not snowflaked Customer Type • Customer Type Cd • Customer Type Desc Customer • Customer Cd • Customer Name • Customer Type Cd Date • Order Date • Week • Month • Year Order Fact • Customer Cd • Sales Rep Cd • Product Cd • Order Date • Order_Qty • Order_Line_Value Sales Rep • Sales Rep Cd • Sales Rep Name • Sales Area Cd Product Line • Product Line Cd • Product Line Name Product Type • Product Type Cd • Product Type Desc • Product_Line Cd Product • Product Cd • Product Name • Product Type Cd Sales Area • Sales Area Cd • Sales Area Desc 22 May 2017 Parent-Child Schema Customer • Customer CdCustomer • • Customer Name Name • Customer • Customer Parent Cd • Customer Parent • a parentNot • child relationship • • Order OrderFact Fact • Cd ••Customer ••Sales Rep Cd ••Product Cd ••Order Date Order Qty ••Order_Qty Order Line Value ••Order_Line_Value Organization • Sales Staff • Sales Rep Name • Sales Rep Cd • Sales Rep Parent • Sales Rep Name • Sales Rep Parent Cd Date • Order Date • Week • Month • Year Product • • Product Name • Product Parent Product • Product Cd • Product Name • Product Parent Cd 22 May 2017 Dimension Builds The main tasks required to define and use a Conformed dimension are . Create a primary reference structure. . Deliver the primary reference structure using a dimension build. . Create a secondary reference structure using the dimension data delivered for the primary reference structure. . Associate the secondary reference structure with a fact build. 22 May 2017 Module 6 Job Streams 22 May 2017 Job Control and the Data Warehouse The basic processes of the data warehouse require automation for the warehouse to function effectively. Raw Data Target Data Mart Automation of: • Data Extraction • Data Transformation • Data Loading • Exception/Error Handling • Logging/Notification 22 May 2017 What Is a JobStream? • A JobStream is a DecisionStream component used to coordinate data mart management tasks. • The JobStream Visualization pane helps to understand the JobStream workflow. 22 May 2017 Characteristics of JobStreams • A JobStream is associated to a single catalog. • Nodes are processed in series or in parallel. • A JobStream is aware only of objects in its catalog but can call system commands (for example, databuild.exe) that can reference other catalogs. Catalog-specific JobStreams contained in separate folder Nodes within a single JobStream 22 May 2017 Add a JobStream to a Catalog JobStreams are added to a catalog and are specific to that catalog. Enter a name and other properties for the JobStream 22 May 2017 Add Variables to a JobStream Define variables to use when creating procedure, condition, and SQL nodes. The COUNTER JobStream variable is used in the Counter Test node 22 May 2017 Main Components of a JobStream Build SQL Procedure Condition Job a fact or dimension build to be executed SQL statement(s) to be executed functions, variables, or actions to be performed tests conditions for branching to other nodes a container for one or more nodes that can include nested jobs 22 May 2017 Build Node Add a fact or dimension build node to a JobStream to automate the extract-transform-load process. Fact build that uses conformed dimensions Conformed dimension builds 22 May 2017 SQL Node Use an SQL node to include SQL statement(s) to be executed during the job control process. Database referenced by the SQL SQL statements to create indexes on a fact table previously loaded by the JobStream 22 May 2017 Procedure Node Use a Procedure Node to coordinate processing around a build. The Execution Log procedure writes a message to the log file using the LogMsg function 22 May 2017 Result Variable • All job nodes (except condition nodes) set the Boolean variable RESULT to indicate success or failure. • Condition nodes check RESULT by default. • A node may deliver its result to another variable if specified. 22 May 2017 Condition Node Use a condition node to test a condition before executing another JobStream component. If the condition specified in Counter Test is False, then End Automation Report executes 22 May 2017 Job Node Use a job node to break large jobs into a series of smaller jobs. If the FactBuild job node is reached, a separate process will run that contains its own logic 22 May 2017 Link JobStream Nodes Each node in a JobStream process must be linked to at least one other node. The Drop Indexes SQL node will be completed BEFORE the SalesFact build node that it is linked to 22 May 2017 Execute Nodes Sequentially or In Parallel These nodes are run in parallel These nodes are run sequentially 22 May 2017 Action on Failure You can specify what DecisionStream does when a node fails. CONTINUE: Continue processing the next node TERMINATE: Terminate the current flow--continue with the next flow ABORT: Stop ALL processing immediately 22 May 2017 Run a JobStream 22 May 2017 Module 7 SQL Text Designer 22 May 2017 SQLTXT Designer You use the SQLTXT Designer to configure the SQLTXT definition files used within DecisionStream to access text files as though they are in an SQL database. SQLTXT Definition Files A SQLTXT definition file (.def) defines a virtual database of text file tables. It associates each Table with a text file in the database directory. It also specifies the structure of the tables. Steps to create a SQLTXT definition file 1. In the SQLTXT Designer, from the File menu, click New. 2. From the File menu, click Save As. 3. In the File name box, type a name for the file. Note: The definition file must have the extension .def. 4. Click Save. 22 May 2017 SQLTXT Designer Steps Import Delimited Records with Delimited Columns 1. Open the appropriate SQLTXT database. 2. From the Edit menu, click Import Table Data. The Import SQLTXT Table Data File dialog box appears. 3. Select the text file to import, and click Open. 4. Click Delimited records. 5. In the Record delimiter box, click the delimiter used to separate individual records in the file. 6. In the Number of header rows box, enter the number of header rows in the file. The SQLTXT Designer uses these header rows to name the columns. 7. In the Decimal symbol box, type a single character that is used for the decimal point in numeric values. 22 May 2017 SQLTXT Designer 8. In the Character set box, enter the encoding used in the text file. 9. In the Digit grouping symbol box, type the character that is used for the thousands separator in numeric values. 10. In the Preview rows box, enter the number of rows that you want to view in the wizard. 11. Click Next. 12. Select Delimited columns. 13. In the Delimiter type box, click the delimiter used to separate the columns in the table. 14. In the Quote character box, click the quotation character used to enclose individual columns within a record. 15. Click Next. 22 May 2017 SQLTXT Designer DecisionStream automatically names and attempts to specify the data type of each column. You can override these selections. 16. To rename a column, click the column name and type a new name. 17. To change a data type, double-click the appropriate column, and in the Data type dialog box, click a different data type. If you select NUMBER, you must also specify the number of digits expected after the decimal point in the Scale box. If you select DATE, you must also specify the format of the date in the Format box. 18. Click OK to close the Data type dialog box. 19. Click Process All Records to ensure that all record lengths and data types are correct. 20. Click Finish. The wizard adds the table to the database, and the SQLTXT Designer appears. 22 May 2017 SQLTXT Designer Steps to define a table manually in SQLTEXT Designer: 1. Click the Database icon. 2. From the Edit menu, click Insert Table. An empty table appears in the tree. 3. Click the new table and, from the Edit menu, click Rename. 4. Type the new name for the table. 5. In the Header lines text box, type the number of header rows present in the file. 6. In the Trailer lines text box, type the number of trailer lines in the file. SQLTXT ignores these rows when it accesses data in the file. 7. In the Character set box, enter the encoding used in the text file. You can click an encoding type from the list, or type the code page into the box. 22 May 2017 SQLTXT Designer 8. In the Action on row error box, click the action to apply when faulty data is read. 9. Select or clear the Remove trailing whitespace from fields check box as appropriate. 10. Specify whether columns in the input text file are of fixed length or delimited . 11. In the Record delimiter box, click the delimiter used to separate individual records. 12. If you selected Delimited records, in the Column delimiter box, specify the delimiter used to separate the columns in the table. 13. If you selected Delimited records, click the quotation character used to enclose individual columns within a record. 14. If you selected Delimited records, select the Ignore extra columns check box to ignore any excess fields. 22 May 2017 SQLTXT Designer Adding Columns to the table manually: 1. Click the table to which you want to add a column. 2. From the Edit menu, click Insert Column. A new column appears beneath the selected table. 3. In the Data type box, click the data type for the column, and specify appropriate length for that column. 4. If the column can accept null values, select the Nullable check box. 5. If the column forms part of the primary key of the table, select the Key check box. 22 May 2017 SQL Term Creating an SQL Query (SELECT Statement) 1. From the Tools menu, click SQLTerm. 2. In the box at the top of the SQLTerm window, click the database with which you want to work. 3. Use the Cognos SQL check box to specify whether you want to use the native SQL or Cognos SQL for the database you are accessing when you construct the SQL statement. Select or clear the check box as appropriate. Tip: Using Cognos SQL makes it easier to port your project between database platforms. Note: The default for the Cognos SQL check box is determined by whether you selected the Cognos SQL check box in the Connection Properties dialog box. 22 May 2017 SQL Term 4. In the Database Objects pane, expand the tree, and then click the table or column for which you want to add a SELECT statement. 5. To generate the SQL statement use one of the following methods: • Right-click, and then click either Add Table Select Statement, or Add Column Select Statement. • Click the Source SQL tab. Hold down Ctrl and drag the table or column you selected in the Database Objects pane to the Source SQL tab. DecisionStream clears any existing query that may be on the Source SQL tab and generates the SQL SELECT statement for the selected table or column. 6. You are now ready to execute the SQL statement. 22 May 2017 Module 8 Command Line interface 22 May 2017 Command Line Interface Executing a Fact build from a DecisionStream Catalog DATABUILD -c<catdb_driver> <catdb_items> <databuild_name>[<options>] Where <catdb_driver> The database driver that DecisionStream must use to connect to the catalog. <catdb_items> The connection string that DecisionStream must pass to the database driver to connect to the catalog. <databuild_name> The name of the build that you want to execute. When you invoke DATABUILD with this syntax, it executes the named build that resides in the named catalog. Example DATABUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" US_Sales This command invokes DATABUILD to execute the build named US_Sales. To retrieve the Build definition, DATABUILD connects to the ODBC data source named 'SalesCat' as user Sales', with password 'ds123'. 22 May 2017 Command Line Interface To Execute a Dimension Build from a Catalog DIMBUILD -c<catdb_driver> <catdb_items> <dimbuild> [<options>] Where <catdb_driver> The database driver that DecisionStream must use to connect to the catalog. <catdb_items> The connection string that DecisionStream must pass to the database driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS', to a catalog that resides in an ODBC data source named “Sample”, you would specify: "DSN=Sample;UID=DBA;PWD=PASS" <dimbuild> The name of the dimension build that you want to execute. Example DIMBUILD -cODBC "DSN=SalesCat;UID=Sales;PWD=ds123" Period The command above invokes DIMBUILD to execute the dimension build named Period. To retrieve the dimension build definition, DIMBUILD connects to the ODBC data source named 'SalesCat' as user 'Sales', with password 'ds123'. 22 May 2017 Command Line Interface RUNDSJOB This command executes a JobStream from a DecisionStream catalog. Syntax RUNDSJOB <catdb_driver> <catdb_items> <jobstream> [<options>] <options> ::= [-e<runenv>] [-j<subjob>] [-A<dbalias_file>] [-C] [-L<logfile>] [-P] [-O] [-R] {-V<name>[=<value>]} Where <catdb_driver> The database driver that DecisionStream must use to connect to the catalog that contains the JobStream. <catdb_items> The connection string that DecisionStream must pass to the database driver to connect to the catalog. For example, to connect as user 'DBA', with Password 'PASS', to a catalog that resides in an ODBC data source named “Sample”, you would specify:"DSN=Sample;UID=DBA;PWD=PASS" <jobstream> The name of the JobStream that DecisionStream must execute. -e Denotes the run environment for the JobStream. 22 May 2017 Command Line Interface RUNDSJOB – Continue … <runenv> The run environment that DecisionStream must apply to identify the correct database alias. -j Denotes the subjob of the JobStream that DecisionStream should execute. <subjob> JobStreams can nest jobs. This option enables you to execute a job that is nested within the JobStream. -A Denotes the text file that contains the database alias definitions that DecisionStream should use for the JobStream. <dbalias_file> The name and path of the text file that contains database alias definitions that DecisionStream must use for the build. -C Executes the JobStream in check-only mode. That is, DecisionStream executes the JobStream but does not deliver any data. -L Causes DecisionStream to log JobStream progress to the specified file. <logfile> The name and path of a text file to which DecisionStream should log build progress. -P Causes DecisionStream to pause at the end of JobStream execution and to prompt for a keystroke before terminating. 22 May 2017 Command Line Interface RUNDSJOB – Continue … -O Causes DecisionStream to execute any builds in the JobStream in object creation mode. That is, DecisionStream executes the builds and creates any required fact and dimension tables and metadata structures, but does not deliver data to them. -R Causes DecisionStream to restart the JobStream at the point of a previous interruption the JobStream restarts at the earliest node that did not successfully complete. -V Denotes that the parameter is a named variable and value pair. <name> The name of a variable that DecisionStream must use during item substitution. <value> The value that the corresponding named variable takes. Example RUNDSJOB ODBC "DSN=Sales;UID=admin" example_job -jbuild_fiscal -R This example invokes RUNDSJOB to restart the subjob named 'build_fiscal' that is nested in The job named 'example_job'. The DecisionStream catalog resides in the ODBC data source Named 'Sales', to which DecisionStream connects as user 'admin' with no password. 22 May 2017 Command Line Interface CATEXP This command exports a catalog item to a DecisionStream file-based specification. Syntax CATEXP <catdb_driver> <catdb_items> <export_file> <comp_type> [<comp_name>] [-a] {-V<name>[=<value>]} <comp_type> ::= A|B|C|D|F|H|J|L|S|T Where <catdb_driver> The database driver that DecisionStream must use to connect to the catalog. <catdb_items> The connection string that DecisionStream must pass to the database driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS', to a catalog that resides in an ODBC data source named 'Sample', you would specify: "DSN=Sample;UID=DBA;PWD=PASS“ <export_file> The path and name of the file to which you want to export the catalog component. <comp_type> The type of component that you want to export. 22 May 2017 Command Line Interface CATEXP – Continue … -a Causes DecisionStream to append the exported component to the named file. <name> The name of a variable. <value> The value that the associated variable takes. <comp_type> The type of component that you want to export. The comp_type values have the following meanings: 22 May 2017 Command Line Interface CATIMP This command imports a text-based component definition into a DecisionStream catalog. Syntax CATIMP <catdb_driver> <catdb_items> <import_filename> <filetype> [<comp_name> [<caption>[<notes>]]] [-d<dbalias>][-o] [-I] {-V<name>[=<value>]} <filetype> ::= B|C|D|F|J|R|S Where <catdb_driver> The database driver that DecisionStream must use to connect to the catalog. <catdb_items> The connection string that DecisionStream must pass to the database driver to connect to the catalog. For example, to connect as user 'DBA', with password 'PASS', to a catalog that resides in an ODBC data source named 'Sample', you would specify: "DSN=Sample;UID=DBA;PWD=PASS“ <import_filename> The path and name of the text file that defines the component. <filetype> The type of component that you want to export. 22 May 2017 Command Line Interface CATIMP – Continue… <comp_name> The name of the individual component that you want to import. This is optional for reference structures, aliases, and functions, but mandatory for other component types. <caption> An optional caption, or business name, that DecisionStream should use for the item for display or presentation purposes. <notes> Optional notes that appear within DecisionStream Designer. 22 May 2017 Command Line Interface CATIMP – Continue… Example CATIMP ODBC "DSN=SalesCat" SalesBld.bld B US_Sales "US Sales -o This command invokes CATIMP to import the build US_Sales from the text file Sales.bld, into the catalog that resides within the ODBC data source named 'SalesCat', overwriting any build with the same name. Within the catalog, CATIMP gives the build US_Sales the caption 'US Sales'. CATIMP connects to the named data source as the default user, with no password. 22 May 2017 Command Line Interface CATIMP – Continue… Order of Import Because DecisionStream validates objects that reside in a catalog, you must import text Definitions in dependency order. For example, do not import a build until the catalog contains the collections and all the aliases and hierarchies that it uses. Normally, you should import text definitions in this order: 1. Aliases 2. User-defined functions 3. Dimension table templates 4. Reference structures (hierarchies and lookups) 5. Collections 6. Builds and dimension builds 7. JobStreams 22 May 2017 Definitions • Build This is a DecisionStream Unit of work that goes through the steps of Acquisition, Transformation, and Delivery. There are two types of build, fact builds and dimension builds. • Collection When DecisionStream is driven from a text file specification, the transformations are split into a build and a collection. The build deals with the delivery of data and the collection deals with the sources of data and the transformation process. • Component A fact build, dimension build, collection, hierarchy, lookup jobstream, user-defined function, template or connection in a catalog. 22 May 2017 Module 9 Definations 22 May 2017 Definitions • Condition Node A condition node provides a branching mechanism between nodes for conditional execution. Each condition node can have many nodes linking to it, but only two output links (True, and False). • Connection All the information required to make a connection to a database. The actual information required varies according to DBMS and connection method. • Data Mart A database that holds data for analysis. It usually implies that the data and its organization is of limited scope or is targeted at a particular group of users and business subjects, for example orders. • Data Source Somewhere from which the DecisionStream can acquire data. 22 May 2017 Definitions • DataStream A DecisionStream object that gathers together all the data sources from which the build or reference structure acquired data. Data streams merge these sources to present a consider whole to other DecisionStream objects. • Designer DecisionStream Designer is the Windows software that you use to produce build specifications and Dimensional Frameworks. • Dimension A Key Component of a business by which you want to analyze your data. For example, the products that your company sells and the retail outlets in which these products are sold. • Dimension Build A Decision Stream build process that delivers only dimension data. 22 May 2017 Definitions • Dimension Template A dimension template lists the columns that may exists in a table of dimension data, together with the behaviour of each column. • Domain Type It is the domain that is used to deliver dimension data at the end of a build. The two types are Dynamic ( where the domain is build up from data sources as rows are processed) and Reference (where the reference object associated with the dimension is taken as the domain) • Factbuild A DecisionStream build that delivers fact data. May optionally deliver dimension data, and metadata, allowing a private datamart to be built from within a single build. • Foster Parent An artificially introduced member that acts as a parent for members that either have no defined parent or whose defined parent cannot be found at the next hierarchy level. 22 May 2017 Definitions • JobStream A DecisionDtream object that combines other DecisionStream object to form a logical process. • Job Node A job node is actually a JobStream within a JobStream. This allows larger jobs to be broken down into a series of smaller jobs. When DecisionStream encounters a job node it processes all the steps within the sub job, and only when these steps are complete, does it move on to the next node in the sequence. • Leaf Node In a hierarchy, a member that has no children. • Lookup A simple reference object that consists of members that are not arranged into hierarchical levels. 22 May 2017 Definitions • Mapping The relationship between columns (from an SQL query) and the DecisionStream objects to which they relate. DecisionStream has mappings that relate data sources to transformation model elements and structure data columns to level attributes. • Message Frequency The rate at which DecisionStream generates messages as it executes a build. Input message frequency refers to how many data rows DecisionStream acquires between messages. Output message frequency refers to how many data rows DecisionStream delivers between messages. • Metadata Data that describe the delivered data. Typically, metadata defines the data structure and the domain of each dimension. It can also provide aliases that reporting tools can use to provide meaningful descriptions. 22 May 2017 Definitions • Never Output An indication on a transformation model element that DecisionStream must not make the element available for delivery. • Object Creation Mode A build execution mode in which fact tables, dimension tables, and metadata are created but no fact or dimension data is delivered. • Output Filter A filter that can be specified in a delivery module to restrict delivery of rows to the target system. The output filter takes any expression that evaluates to TRUE or FALSE. • Output Level A level of hierarchy at which DecisionStream will output data. Output data may be generated by aggregation. 22 May 2017 Definitions • Pivot To move a dimension from on axis to another. For example, consider a relational table that has a single column (called product) that holds all product values. Assume that this has one column fro each business measure (units, revenue, etc.) The product column could be pivoted to produce one column for each product and the business measures pivoted to produce one row for each measure. • Procedure Node A procedure node can contain one or more DecisionStream functions or statements. This node can be used to coordinate processing around builds for such activities as checking for input files, sending mail messages and alerts, and generating custom logging and auditing messages. • Ragged Hierarchy A hierarchy that has leaf nodes at more than one level. Also called an unbalanced hierarchy. 22 May 2017 Definitions • Reference Explorer A DecisionStream object that gathers together all reference structures that relate to a particular business dimension. • Reject File A text file in which DecisionStream stores records that the engine rejects. In these files, each rejected record has the reason for rejection as its first field. • Sample Rate When reading from a data source, DecisionStream can apply a sample rate so that it does not process every row. For example, a sample rate of 10 results in DecisionStream processing every tenth row and ignoring the intermediate rows. • Slowly Changing Dimension A business dimension in which one or more non-key values may change without a corresponding change in the key values. 22 May 2017 Definitions • SQL Node Use an SQL node to enter a single SQL statement that DecisionStream performs when the SQL node is reached in the JobStream flow. DecisionStream automatically commits the successful SQL statements, but roll back if the statement fails. • SQLTerm A terminal monitor application that provides a common interface to the various relational database products that DecisionStream supports. • SQLTXT Designer It is a TOOL which enables text files to be described in a way that subsequently allows SQLTerm to access the data that the text files hold. • Transformation The re-expression of a stream of data in some different format. Typically the merging and aggregation of source data to provide delivery elements. 22 May 2017 Definitions • Star Schema A star schema represents the dimension in a single table. Each level of the delivered hierarchy is represented by one or more columns of this table. In order to use the Dimension Build wizard to create a dimension build that supports slowly changing dimensions, you must use this schema type. • Surrogate Key Surrogate keys are numeric keys that can replace business keys in the context of a data mart. Whereas the natural ID may be text-based, the surrogate key is always an integer. • Tree Pane The left pane of the Designer window where DecisionStream arranges the objects of the catalog in a tree-like hierarchical fashion. • Transformation Model Collection of dimension, measure, attribute and derivation elements in a fact build. 22 May 2017 Thank You 22 May 2017