Download SCT Operational Data Store / Banner to the ODS Implementation

SCT Banner to the
SCT Operational Data Store
Implementation Guide
October 2005
Release 2.2
What can we help you achieve?
Confidential Business Information
This documentation is proprietary information of SunGard SCT and is not to be copied, reproduced, lent or disposed of, nor used for
any purpose other than that for which it is specifically provided without the written permission of SunGard SCT.
Prepared By:
SunGard SCT
4 Country View Road
Malvern, Pennsylvania 19355
United States of America
© SunGard 2004-2005. All rights reserved. The unauthorized possession, use, reproduction, distribution, display or disclosure of this
material or the information contained herein is prohibited.
In preparing and providing this publication, SunGard SCT is not rendering legal, accounting, or other similar professional services.
SunGard SCT makes no claims that an institution's use of this publication or the software for which it is provided will insure compliance
with applicable federal or state laws, rules, or regulations. Each organization should seek legal, accounting and other similar
professional services from competent providers of the organization's own choosing.
SunGard, the SunGard logo, SCT, Banner, Campus Pipeline, Luminis, PowerCAMPUS, SCT fsaATLAS, SCT Matrix, SCT Plus, SCT
OnSite and SCT PocketRecruiter are trademarks or registered trademarks of SunGard Data Systems Inc. or its subsidiaries in the U.S.
and other countries. All other trade names are trademarks or registered trademarks of their respective holders.
Implementation Guide
SCT Banner to the Operational Data Store
1 Overview
Triggers and Change Tables . . . . .
Composite Views . . . . . . . . . .
ODS Composite Tables . . . . . . .
Extract, Transform and Load Process
ODS Reporting Views . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1-3
1-5
1-6
1-7
1-7
Prerequisite Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
2 Composite Views
iii
Table of Contents
This page intentionally left blank
iv
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential
1
Chapter 1
Overview
Overview
The Operational Data Store (ODS) is designed to extract information from a source
system such as SCT Banner, and reformat the tables into a simplified set of tables to
allow for quick and easy reporting. The concept is based on the SCT Banner
Object:Access approach of grouping complex and normalized source tables into a
series of more simplified, denormalized tables that are grouped by concept as in the
Personal Demographic data example below:
SCT Banner tables of
personal information
SPRIDEN
SPBPERS
SPRADDER
e
SCT Banner view of
personal information
AS_PERSON
To properly access the data from the SCT Banner tables above for reporting, you
need to understand the rules used to store the data in each table, as well as the rules
used to properly join the tables. These tables are some of the most commonly used
tables in SCT Banner for extracting person related information. Some examples
appear below:
Examples:
SPRIDEN
•
Must be checked for a value of null to retrieve the most current record
•
ENTITY_IND must be checked for a Person or Corporation (though for
Advancement this is Person or Organization)
•
Must be “driving table” in FROM clause
SPBPERS
October 2005
Confidential
•
Join with SPRIDEN via PIDM column
•
Outer Join must be used or omitted based on need
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
1-1
1
Overview
SPRADDR
•
Join with SPRIDEN via PIDM column
•
Outer Join must be used or omitted based on need
•
STATUS_IND must be checked for active record
•
FROM_DATE and TO_DATE must be checked for current date range allowing for
NULL values to be included
•
Highest or Lowest SEQ_NO within ADDRESS_TYPE must be used based on need
•
ADDRESS_TYPE must be checked against a hierarchy of values to ensure that no
more than one address is retrieved
You can access SCT Banner information without understanding the complexities of
the data structure because you can retrieve the data from the view. This method
provides basic data access, but performance and non-SCT Banner data integration
issues are not addressed within the SCT Banner database.
Using the Oracle Warehouse Builder (OWB) ETL tool, triggers, and new data
change tables in SCT Banner, the ODS extracts data from SCT Banner and loads it
into a separate database schema. Indexes are added to the ODS tables as needed to
enable the information to be accessed. The ODS is updated as needed (Incremental
Refresh) to make additions, deletions or changes in the SCT Banner database. The
data is then presented in a view similar. A diagram of the process appears below:
Loading the ODS
PERSON
Source
Table
PERSON
Composite
Table
Trigger
CHANGE
Table
ID
Source
Table
PL/SQL ETL
OWB
Mappings
Reporting
Views
Trigger
Changes (create, update, delete)
to source tables launch triggers
that update appropriate change
tables.
PERSON
Composite
View
ODS install process builds
base and reporting views.
Incremental refresh process
updates these views.
End users build reports
from ODS reporting views.
ETL – When OWB mappings
run, they join Oracle source
composite views with related
change tables and update ODS
composite tables.
Source Oracle Database
1-2
PERSON
Base
View
ODS
SCT Operational Data Warehouse Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential
1
Overview
Triggers and Change Tables
Triggers
A single database triggers exist on each source table, excluding validation tables.
Triggers are created for all tables used in a view, including function tables. They are
created in the schema owner of the associated change table.
Each trigger identifies DML (Data Manipulation Language) activity on the table.
When a change is made to an SCT Banner table, that change is loaded into the ODS.
The change launches a trigger. The trigger calls a stored PL/SQL procedure which
updates the appropriate change tables to reflect the change in the source table. The
triggers flag changes on the SCT Banner tables and update the appropriate change
tables.
Triggers are created on the actual SCT Banner tables that provide source data for
the ODS. The triggers are not delivered with the baseline SCT Banner applications.
Trigger Packages
Trigger packages manage the trigger procedures. There is one procedure for each
change table with each procedure managing a unique index on the change table.
There is one package per product area within the ODS such as Student, Human
Resources, Finance, Financial Aid, Advancement. BANINST1 owns the trigger
packages.
As data is entered into SCT Banner, it is typically processed one row at a time. For
each field being entered, the data is verified for field syntax, such as date or numeric
formats, and fields requiring additional verification against rule tables. After the
values are properly checked, the data is committed (or written) to the database table
that will ‘house’ the information. During the ‘commit’ action, any Oracle triggers
on the database table being written to are ‘fired’ and additional, but separate, logic
is executed based on the parameters of the trigger (such as Before Insert, After
Insert, etc.). The ODS has built and enabled triggers on all SCT Banner tables that
house information that is used in the ODS. Therefore, when one of the SCT Banner
ODS triggers is ‘fired’, the trigger inserts the keys of the data being changed into the
change tables along with a DML indicator. The existence of these rows in the change
tables tells the ODS that SCT Banner has data waiting to be retrieved.
Note: The change tables only maintain the most recent database activity for a
row of information for a specific key. When multiple actions occur
against the same SCT Banner database table and row, only the last action
is represented in the change table. This allows the replication process to
work faster, and decreases the amount of data captured in the change
tables.
October 2005
Confidential
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
1-3
1
Overview
Change Tables
Change tables reflect DML activity for specific SCT Banner tables, but are also used
when multiple tables use the same key.
Example:
The use of the SPRPCHG table that stores DML activities for the Hold and the
Person composite views.
Change tables are owned by their respective product schemas, and are identified
using standard SCT Banner table naming conventions. The column names start with
the seven-character prefix of the table name. All columns in each of the change
tables are identical with the exception of the key columns. Here, the key columns
represent the product/database tables they are accessing, and also represent the
keys that the ODS uses when records change. All change tables are suffixed by
‘CHG’.
The columns that comprise the change table are the key columns relative to the
composite view(s) it supports, along with the TABLE_NAME and the PROCESS_ID
columns. The last two columns allow inserts into the table with a null PROCESS_ID
by updates to SCT Banner that take place during Incremental Refresh. Since the
ODS processes and deletes all rows in the change tables with a NOT NULL
PROCESS_ID, the null value allows the row to stay until the next update. This ensures
that it is not bypassed or inadvertently deleted.
Typically, a second index is created in the format of TABLE_NAME, PROCESS_ID, and
RECORD_ACTION columns.
Example:
SPRPCHG – Change table for PIDM related SCT Banner tables
1-4
Column Name
Data Type
Column Comment
SPRPCHG_TABLE_NAME
VARCHAR2(30)
Used to identify which composite view (and/or ODS
table) is being populated by this specific row of data.
SPRPCHG_PIDM
NUMBER
The change table needs to hold as many keys as are
required to manage DELETE and UPDATE of
information into the ODS. Keys do not need to
identify a unique row, but must maintain some fields
for comparison.
SPRPCHG_RECORD_
ACTION
VARCHAR2(1)
Stores the last DML action for the key combination (I,
U, or D).
SCT Operational Data Warehouse Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential
1
Overview
Column Name
Data Type
Column Comment
SPRPCHG_PROCESS_ID
VARCHAR2(30)
Updated by the ODS procedure
UPDATE_CHANGE_TABLE which inserts non-null values
to flag which rows are being processed during the
incremental refresh process. This allows inserts to take
place into the change table while replication is also
taking place.
SPRPCHG_ACTIVITY_
DATE
DATE
Reflects the actual date of that the rows was last
inserted and/or updated.
Change Table Triggers
The ODS maintains triggers on all SCT Banner tables used to incrementally refresh
data into the ODS. Although the triggers are enabled on the actual SCT Banner
tables, they are referred to as ‘change table triggers’ because they populate the SCT
Banner change tables with DML information. The trigger inserts rows of
information in one or more change tables by invoking a procedure that packages all
trigger insert actions for the ODS change tables.
The triggers use basic logic except that the Exception routines allow for continued
processing when encountering a DUP_VAL_ON_INDEX condition. This condition is
occurs when a row of data exists within the change table for the table’s unique index.
When encountered, the procedure updates rather than inserts the information in
the change table by overlaying the DML activity and the activity date. This action
causes only the most recent DML activity to be stored in the change table.
All triggers are owned and maintained within the product schema of the table to
which the triggers are added. For example, SATURN would own Student Triggers,
etc.
•
Trigger Procedures
•
Each SCT Banner product has a procedure built for it that manages all
change table triggers for that product area. For example, GOKODST for
General, SOKODST for SATURN, etc.
•
The triggers are owned by the BANINST1 schema.
•
The names for each procedure follow SCT Banner standard naming
conventions.
Composite Views
Composite views exist in SCT Banner and usually match the composite tables that
exist in the ODS. During ETL, when you perform a refresh of ODS data, the
composite views are joined with the appropriate change tables and updated with the
changed information.
October 2005
Confidential
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
1-5
1
Overview
In some cases, functions are used to calculate new data that is created from SCT
Banner data and loaded into the ODS. The composite views use packages to manage
the functions built on the SCT Banner database. There is one function package for
each ODS module of information, i.e., Student, Finance, Advancement, etc.
Composite views represent a composite (mixture) of the tables selected from SCT
Banner and allow for a single piece of data to be extracted row-by-row, with all the
business logic included in the view itself. These views use Object:Access standards.
The column names are generic so that they can be used by all SunGard SCT product
lines. Therefore, names familiar to SCT Banner clients may be more generic than
they are used to. For example, Term becomes Academic Period, PIDM becomes UID
(unique ID).
The views can be used for reporting in SCT Banner. But, they were designed to be
extracted and loaded into the ODS, joined to SCT Banner change tables, and to
become the Incremental Refresh data extraction view.
Views are created and maintained in the BANINST1 schema. SELECT access to
them is only granted to the ODSMGR schema in the SCT Banner database. These
views can only be accessed via the ODS through database links that connect from the
ODS ODSMGR schema to the BANNER ODSMGR schema.
ODS Composite Tables
Composite tables are the tables within the ODS that are loaded with data from SCT
Banner. These tables match element-for-element to the SCT Banner composite
views. The composite tables are populated during the initial install process. They are
also updated during the incremental refresh process.
•
Denormalized tables are used to store “conceptual” structures of data.
•
Normalized tables are used for quick data filtering or to include unlimited
repeating values.
Slotted Tables
•
Slotted are used to denormalize predefined Repeating Concepts (normalized
tables.)
•
Populated via packages sometimes using MGRSDAX rules.
Some GTVSDAX rules, but not values, are duplicated in the initial population of
MGRSDAX. You should use the Administrative UI to add or modify MGRSDAX
rules' values to meet your institution’s needs.
1-6
SCT Operational Data Warehouse Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential
1 Overview
Prerequisite Tasks
Extract, Transform and Load Process
The ODS, and its associated data replication process, simplified reporting by
replicating data onto a separate database apart from the production environment.
This process is called the Extract, Transform, and Load (ETL) process.
ETL extracts data from the SCT Banner database, transform the data, and loads it
into the ODS. All ETL activities are performed from within Oracle PL/SQL
packages, and deployed into the ODS database schema. These packages are
scheduled and run via the DBMS_JOBS queue in Oracle. You can submit and
monitor the jobs using the Information Access Administrative User Interface.
Typically referred to as ‘mappings’, the packages, when executed, delete, update
and load data from SCT Banner to the composite tables based on the type of
mappings executed.
Note: The SCT Banner and ODS database instances should reside on separate
servers and connect through a database link.
ODS Reporting Views
Data from each ODS composite table is presented in one or more reporting views.
The ODS reporting views are the views that your users use to create reports within
the ODS. Users point their report writing tool at these views and build reports. Base
views can also be used as reporting views.
The SCT Datamart 1.0 product Object:Access views were recreated as additional
reporting views for clients migrating reports written using the Object:Access views in
the Datamart, or from SCT Banner into the ODS. While SCT will continue to
support these views, they will only be enhanced to support migration, therefore, use
of the views to create new reports is not recommended.
These Object:Access views are easy to identify because they use the same standard
naming conventions as the Object:Access views delivered with the SCT Banner
products.
Prerequisite Tasks
The prerequisites below must be completed for the SCT Banner General and
Finance products before the ODS tables are loaded initially. No prerequisites are
required for the remaining SCT Banner products.
October 2005
Confidential
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
1-7
1 Overview
Prerequisite Tasks
Task 1—Define General Preferences Address Type
The MST_ADDRESS composite table stores a person's preferred address information
from the SCT Banner SPRADDR table. Since this information is rule based, the
Address Type value needs to be defined in the SCT Banner GTVSDAX form so that
the SCT Banner view AS_PERSON knows which value to use when selecting the
address records. All values entered into the GTVSDAX form should stay exactly as
listed below except for the external code. This value should be changed to match
the client specific address type from the STVATYP table that defines a preferred
address.
Note: Only one address type can be specified for the preferred address.
Example:
External Code
Internal Code
Internal Code
Sequence Number
Internal Code Group
PR
PREFADDR
1
ADDRESS
Caution: You must alter and/or enter your institution’s preferred address type
before starting the ODS load and incremental refresh process. If this
parameter is not set up, null values will appear in the preferred address
fields.
Task 2—Define Fiscal Years for Transaction History Loads in Finance
The MFT_TRANSACTION_HISTORY composite table in the ODS stores detailed history
information from the SCT Banner Finance transaction tables FGBTRND and
FGBTRNH. These tables are typically very large and take a long time to load. To
improve the performance and allow intermittent commits, the load process is
‘staged’ into five separate load processes. This allows the process to restart if any
problems occur during the load. These load processes are driven by fiscal years.
Which fiscal years are loaded, and to which load mappings they should belong is
defined on the Set Up Parameters page of the Administrative UI. (MTVPARM)
Note: There may be more than one fiscal year defined for each of the load
mappings. Therefore, you may define fiscal year groupings that best suit
your system resources. See the example below:
1-8
SCT Operational Data Warehouse Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential
2
Chapter 2
Composite Views
Composite Views
The composite views gather the SCT Banner source data necessary to
populate and maintain the information stored in the SCT Operational Data
Store (ODS). This source information then updates the information that
resides on the separate ODS database.
Note: Any institution change to a composite view impacts the ODS
maintenance processes.
ODS composite view meta data is also available as published meta data. Use the
following steps to view published composite view meta data reports using the
Administrative UI.
1.
Select Information Access Meta Data from the Information Access
Administrative menu.
2.
Select Operational Data Store. The Operational Data Store Reporting View
Meta Data Reports page opens.
3.
Select the ODS Composite View Meta Data Reports link located at the top
right-hand corner of the page. The Operational Data Store Composite View
Reports page opens.
4.
Select a subject area. The Composite View Meta Data Reports page opens
listing the view name and description.
5.
To display the column details associated with the selected composite view,
select one of the composite views. A description of each field on the report
appears below:
Descriptions / Buttons
Fields
.............. ...................................................
October 2005
Confidential
Description
Business description of the composite view target, including the
key and frequency of data returned by the view.
Target Column
Column name in the composite view target.
Business
Definition
Defines the target column in business terms.
Database Data
Type
Used for formatting purposes when writing reports.
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
2-1
2
Composite Views
Fields
Descriptions / Buttons
.............. ...................................................
Business Data
Type
This field is empty by default. It can be used to store clientspecific data about a given column.
Domain Value
This field is empty by default. It can be used to store clientspecific data about a given column.
Source Name
Source table, FUNCTION, CONSTANT, or CALCULATION
Source Column
Source column name from the source table or view, if the source
is a table or view. Name of the PL/SQl function if the source is
FUNCTION, description of the constant if the source is CONSTANT,
or description of the calculation used if the source is
CALCULATION.
2-2
SCT Operational Data Store Release 2.2
SCT Banner to the Operational Data Store Implementation Guide
October 2005
Confidential