Download GRIN-Global Data Overview document

GRIN-Global Data Overview
(draft)
a work in progress…
Revision Date
June 1, 2010 – [Publish Date]
Thank you for your patience! Any references that include [tbd] (“to be developed”) will be developed –
we promise!
This guide provides …
Comments/Suggestions:
Please contact Marty Reisinger with any suggestions or questions related to this document.
[email protected] // Beltsville # 1.301.504.5439 // Telework # 1.410.666.0100
841007837
Page | 2
Contents
GRIN-Global Data Overview
4
Overview ................................................................................................................................................... 4
SQL Abstraction Layer ............................................................................................................................... 4
Parameter Delimiters ............................................................................................................................ 5
Limit and Offset ..................................................................................................................................... 5
Common Function Standardization ...................................................................................................... 5
Returning Auto-generated Values ........................................................................................................ 5
Data Type Standardization .................................................................................................................... 5
Table Mappings ......................................................................................................................................... 6
Dataviews .................................................................................................................................................. 6
Data Triggers ............................................................................................................................................. 7
Extending the Schema............................................................................................................................... 8
Create a Table Definition ...................................................................................................................... 8
Connect GRIN-Global with the new Table ............................................................................................ 8
Indicate to GRIN-Global How to Display the Table’s Data .................................................................... 8
Creating a Data Trigger ............................................................................................................................. 9
Creating a .NET assembly ...................................................................................................................... 9
Associating a Data Trigger and a Resource ........................................................................................... 9
841007837
Page | 3
GRIN-Global Data Overview
Overview
One overriding design consideration for the GRIN-Global system is to meet each genebank’s unique data
processing requirements. Several fundamental components are used to achieve this flexibility and
facilitate a highly customizable system:
Component
Purpose
Related Database Tables
SQL Abstraction Layer
Modifies SQL statements at
runtime to conform to each
database engine’s specific
“flavor” of SQL
Not applicable
Table Mappings
Caches metadata about the
database schema itself, which is
used by dataview processing
sys_table
sys_table_lang
sys_table_field
sys_table_field_lang
sys_table_relationship
Dataviews
User-editable SQL SELECT
statements are used to display
and modify data
sys_dataview
sys_dataview_lang
sys_dataview_param
sys_dataview_sql
sys_dataview_field
sys_dataview_field_lang
Data Triggers
User-extensible .NET code that
runs whenever data is read or
saved via a dataview
sys_datatrigger
SQL Abstraction Layer
GRIN-Global has a special data layer (GrinGlobal.Core.DataManager) that minimizes the
differences that exist between the various supported database engines. Specifically, it addresses the
following nuances:
 parameter delimiters
 limit and offset
 common function standardization
 returning auto-generated values
 data type standardization
841007837
Page |4
Parameter Delimiters
The following table lists the three parameter delimiters allowed by the four supported database
engines:
Parameter Delimiter
Database Engine
@
SQL Server
?
MySQL
:
PostgreSQL and Oracle
GRIN-Global’s DataManager allows any of the delimiters to be specified and adjusts the SQL query to fit
the current database engine. This feature makes it possible for dataview creators to use any parameter
format with any of the four engines.
Limit and Offset
Limiting query results to a maximum number of records and offsetting to return only a subset of those
records is very useful – and extraordinarily different between the four database engines. DataManager
provides the Limit and Offset properties to negate these differences. The SQL query is formatted in real
time and passed to the database engine, meeting the database engine’s specific requirements.
Common Function Standardization
Functions are standardized to meet the current database engine requirements.
Function
Converted to the Appropriate…
CONCAT()
string concatenation operator(s)
TRIM()
string trimming construct(s)
LEFT()
string truncation construct(s)
CONVERT()
type casting construct(s)
Returning Auto-generated Values
Each database engine has its own unique syntax for INSERT statements to generate a new value to serve
as the primary key for a new row. Instead of requiring a developer to write different INSERT statements
for different database engines, DataManager encapsulates this disparity by manipulating INSERT
statements at runtime to fit the requirements of the database engine currently in use.
Data Type Standardization
Each database engine expects different type names for the same kind of data type. For example, a
column that stores both date and time may be “DATETIME” in one database engine, “TIMESTAMP” in
another, and “DATETIME2” in yet another. DataManager interprets the standard DbType enumeration
values and selects the appropriate one for the current database engine.
841007837
Page |5
Table Mappings
Each database engine (SQL Server, My SQL, Oracle, PostgreSQL) stores information about its tables,
constraints, foreign keys, and indexes differently.
Table and column Information, the “table mappings,” are required for every INSERT, UPDATE, or DELETE
SQL statement issued to the database engine so that validation can be performed for data typing,
maximum length, etc. Since these table mappings are costly to retrieve in real time with some of the
database engines, part of the GRIN-Global database schema includes a place to cache table mapping
information. These table mappings serve the following purposes:
Purpose
Description
Data Updatability
A field in a dataview must correspond to a given table mapping
for that field to be updatable
Security
A permission that restricts down to the row level uses table
mappings to specify the rows to which the permission apply
Language Information
Language-specific Title and Description values are used as
defaults when defining a dataview
Dataviews
GRIN-Global dataviews are a means for dynamically adding new queries to the Curator Tool. A GRINGlobal Dataview is essentially a stored SQL Query with some peripheral information. A dataview consists
of the following information:
SQL Query Definitions
A SQL statement for each of the supported database engines -SQL Server, MySQL, PostgreSQL, and Oracle
SQL Parameter Definitions
For passing user specified values to the SQL queries at run time
Field Mappings
For associating a dataview field with a specific table and column
in the database, so the data in that field can be updated by the
user
Language Information
Language-specific Title and Description for column headings
displayed in the Curator Tool, Search Tool, and Import Wizard
Flags and Settings
Denotes if the dataview is read only, user visible, etc.
841007837
Page |6
Data Triggers
A “native trigger” is a procedural database-engine specific extension to the SQL-99 specification that is
automatically executed in the context of the database engine in response to certain events on a
particular table or view in a database. Since the implementation of native triggers – those defined within
the database itself –varies so greatly among the four supported database engines, GRIN-Global eschews
their use in favor of a generic, .NET-based implementation that can run across all database engines
equally.
These generic, database engine-agnostic modules are called “data triggers” in GRIN-Global. While a
native trigger executes in the context of the database engine, a GRIN-Global data trigger executes in the
context of the middle tier. The middle tier itself is hosted by the GRIN-Global web service, so all data
triggers in GRIN-Global execute outside the current database engine. The data trigger processing code
can be found in the following namespace: GrinGlobal.Business.SecureData Data triggers
have the following characteristics:
Easy to Implement
Minimal implementation entails creating a concrete class of only one
of the following (in the
GrinGlobal.Interface.DataTriggers namespace):
IDataViewReadDataTrigger
IDataViewSaveDataTrigger
ITableReadDataTrigger
ITableSaveDataTrigger
Configurable
The Admin Tool can be used to associate a Data Trigger with a
dataview or a table. The trigger can also be disabled without being
removed for testing and troubleshooting purposes
Powerful
A data trigger is not restricted to affecting only the database – it is
full-fledged .NET code. It could send an email, manipulate an image,
write to a file, generate a report, call a web service, etc.
Granular
Multiple data triggers can be defined for each table or dataview,
meaning each data trigger can perform one specific task instead of
having to perform multiple, possibly unrelated tasks. Also code
within the data trigger itself can be written to do nothing unless the
data meets specific criteria, so row-level granularity is possible.
841007837
Page |7
Extending the Schema
The GRIN-Global schema is partially derived from GRIN, its predecessor. Much of the data tracked by
genebanks world-wide is similar and is included in the base schema, but since most genebanks do things
slightly differently, GRIN-Global makes it relatively easy to add new tables to the schema and have the
GRIN-Global software manipulate its data – with no change to the software itself. The following three
steps are all that are needed:
1. Create a new table definition in the database schema
2. Connect GRIN-Global with the new table
3. Indicate to GRIN-Global how to display the table’s data
Create a Table Definition
GRIN-Global supports multiple database engines. Since applying schema changes to a particular
database engine varies greatly from another one, GRIN-Global does not provide a mechanism for
creating the table definition itself.
Consult the documentation of your particular database engine for details on how to accomplish this
task.
Connect GRIN-Global with the new Table
Inspecting a database schema can take several seconds depending on which database engine GRINGlobal is using. For this reason, all tables must be explicitly “mapped” – cached in the GRIN-Global
system tables themselves – before they can be used by GRIN-Global. This action needs only be done
when schema changes have occurred. The steps to do this are:
1. Open the GRIN-Global Admin Tool.
2. Right-click Table Mappings, select Inspect Schema for Unmapped Tables.
3. Choose which table(s) to map.
4. Click Generate Default Mappings.
Indicate to GRIN-Global How to Display the Table’s Data
After the previous steps’ table mapping has been completed, a dataview can be created to read and
write data to the table. To create a default dataview to see all the data in the new table:
1. Open the GRIN-Global Admin Tool.
2. Right-click Dataviews; select New Dataview.
3. Find the table in the treeview in the left panel; drag the table name to the textbox on the right.
4. Click the Properties tab and provide a unique name for the dataview.
5. Click the Titles tab and provide a language-specific title and description for the dataview.
6. Click Save.
841007837
Page |8
Creating a Data Trigger
Usually a dataview will suffice for basic reading and writing of data. However, there may be instances
when pre- or post-processing will be performed. For example, there may be a business rule that the
curator should be emailed whenever inventory falls below a certain threshold. This functionality is
handled by creating a custom data trigger.
NOTE: Data triggers require the ability to write and compile .NET code. The tools for doing this are
freely available from Microsoft.
At the high level, creating a data trigger consists of:
1. Creating a .NET assembly (.dll file)
2. Associating the assembly with a particular dataview or resource in GRIN-Global
Creating a .NET assembly
The following basic steps are required to create a .NET assembly:
1. Open Visual Studio 2005 or greater
2. Create a new Class Library project
3. Add a reference to GrinGlobal.Interface.dll
4. Implement at least one Data Trigger interface
(there are four defined in GrinGlobal.Interface.Dataviews.*)
5. Compile
Associating a Data Trigger and a Resource
Associating a data trigger to a particular resource is similar to the “table mappings” described earlier –
GRIN-Global needs to be given the data trigger assembly and told when it should run the assembly’s
code. This can be done very easily:
1. Open the GRIN-Global Admin Tool
2. Right click Data Triggers; select New Data Trigger
3. Click Add New… to find your assembly file
4. Choose the resource with which to associate the data trigger –a dataview, a table, or neither
(meaning any resource)
5. Select which class contained in the assembly should be loaded for that resource
6. Click Save
841007837
Page |9