* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download GRIN-Global Data Overview document
Survey
Document related concepts
Oracle Database wikipedia , lookup
Concurrency control wikipedia , lookup
Ingres (database) wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Functional Database Model wikipedia , lookup
ContactPoint wikipedia , lookup
Clusterpoint wikipedia , lookup
Transcript
GRIN-Global Data Overview (draft) a work in progress… Revision Date June 1, 2010 – [Publish Date] Thank you for your patience! Any references that include [tbd] (“to be developed”) will be developed – we promise! This guide provides … Comments/Suggestions: Please contact Marty Reisinger with any suggestions or questions related to this document. [email protected] // Beltsville # 1.301.504.5439 // Telework # 1.410.666.0100 841007837 Page | 2 Contents GRIN-Global Data Overview 4 Overview ................................................................................................................................................... 4 SQL Abstraction Layer ............................................................................................................................... 4 Parameter Delimiters ............................................................................................................................ 5 Limit and Offset ..................................................................................................................................... 5 Common Function Standardization ...................................................................................................... 5 Returning Auto-generated Values ........................................................................................................ 5 Data Type Standardization .................................................................................................................... 5 Table Mappings ......................................................................................................................................... 6 Dataviews .................................................................................................................................................. 6 Data Triggers ............................................................................................................................................. 7 Extending the Schema............................................................................................................................... 8 Create a Table Definition ...................................................................................................................... 8 Connect GRIN-Global with the new Table ............................................................................................ 8 Indicate to GRIN-Global How to Display the Table’s Data .................................................................... 8 Creating a Data Trigger ............................................................................................................................. 9 Creating a .NET assembly ...................................................................................................................... 9 Associating a Data Trigger and a Resource ........................................................................................... 9 841007837 Page | 3 GRIN-Global Data Overview Overview One overriding design consideration for the GRIN-Global system is to meet each genebank’s unique data processing requirements. Several fundamental components are used to achieve this flexibility and facilitate a highly customizable system: Component Purpose Related Database Tables SQL Abstraction Layer Modifies SQL statements at runtime to conform to each database engine’s specific “flavor” of SQL Not applicable Table Mappings Caches metadata about the database schema itself, which is used by dataview processing sys_table sys_table_lang sys_table_field sys_table_field_lang sys_table_relationship Dataviews User-editable SQL SELECT statements are used to display and modify data sys_dataview sys_dataview_lang sys_dataview_param sys_dataview_sql sys_dataview_field sys_dataview_field_lang Data Triggers User-extensible .NET code that runs whenever data is read or saved via a dataview sys_datatrigger SQL Abstraction Layer GRIN-Global has a special data layer (GrinGlobal.Core.DataManager) that minimizes the differences that exist between the various supported database engines. Specifically, it addresses the following nuances: parameter delimiters limit and offset common function standardization returning auto-generated values data type standardization 841007837 Page |4 Parameter Delimiters The following table lists the three parameter delimiters allowed by the four supported database engines: Parameter Delimiter Database Engine @ SQL Server ? MySQL : PostgreSQL and Oracle GRIN-Global’s DataManager allows any of the delimiters to be specified and adjusts the SQL query to fit the current database engine. This feature makes it possible for dataview creators to use any parameter format with any of the four engines. Limit and Offset Limiting query results to a maximum number of records and offsetting to return only a subset of those records is very useful – and extraordinarily different between the four database engines. DataManager provides the Limit and Offset properties to negate these differences. The SQL query is formatted in real time and passed to the database engine, meeting the database engine’s specific requirements. Common Function Standardization Functions are standardized to meet the current database engine requirements. Function Converted to the Appropriate… CONCAT() string concatenation operator(s) TRIM() string trimming construct(s) LEFT() string truncation construct(s) CONVERT() type casting construct(s) Returning Auto-generated Values Each database engine has its own unique syntax for INSERT statements to generate a new value to serve as the primary key for a new row. Instead of requiring a developer to write different INSERT statements for different database engines, DataManager encapsulates this disparity by manipulating INSERT statements at runtime to fit the requirements of the database engine currently in use. Data Type Standardization Each database engine expects different type names for the same kind of data type. For example, a column that stores both date and time may be “DATETIME” in one database engine, “TIMESTAMP” in another, and “DATETIME2” in yet another. DataManager interprets the standard DbType enumeration values and selects the appropriate one for the current database engine. 841007837 Page |5 Table Mappings Each database engine (SQL Server, My SQL, Oracle, PostgreSQL) stores information about its tables, constraints, foreign keys, and indexes differently. Table and column Information, the “table mappings,” are required for every INSERT, UPDATE, or DELETE SQL statement issued to the database engine so that validation can be performed for data typing, maximum length, etc. Since these table mappings are costly to retrieve in real time with some of the database engines, part of the GRIN-Global database schema includes a place to cache table mapping information. These table mappings serve the following purposes: Purpose Description Data Updatability A field in a dataview must correspond to a given table mapping for that field to be updatable Security A permission that restricts down to the row level uses table mappings to specify the rows to which the permission apply Language Information Language-specific Title and Description values are used as defaults when defining a dataview Dataviews GRIN-Global dataviews are a means for dynamically adding new queries to the Curator Tool. A GRINGlobal Dataview is essentially a stored SQL Query with some peripheral information. A dataview consists of the following information: SQL Query Definitions A SQL statement for each of the supported database engines -SQL Server, MySQL, PostgreSQL, and Oracle SQL Parameter Definitions For passing user specified values to the SQL queries at run time Field Mappings For associating a dataview field with a specific table and column in the database, so the data in that field can be updated by the user Language Information Language-specific Title and Description for column headings displayed in the Curator Tool, Search Tool, and Import Wizard Flags and Settings Denotes if the dataview is read only, user visible, etc. 841007837 Page |6 Data Triggers A “native trigger” is a procedural database-engine specific extension to the SQL-99 specification that is automatically executed in the context of the database engine in response to certain events on a particular table or view in a database. Since the implementation of native triggers – those defined within the database itself –varies so greatly among the four supported database engines, GRIN-Global eschews their use in favor of a generic, .NET-based implementation that can run across all database engines equally. These generic, database engine-agnostic modules are called “data triggers” in GRIN-Global. While a native trigger executes in the context of the database engine, a GRIN-Global data trigger executes in the context of the middle tier. The middle tier itself is hosted by the GRIN-Global web service, so all data triggers in GRIN-Global execute outside the current database engine. The data trigger processing code can be found in the following namespace: GrinGlobal.Business.SecureData Data triggers have the following characteristics: Easy to Implement Minimal implementation entails creating a concrete class of only one of the following (in the GrinGlobal.Interface.DataTriggers namespace): IDataViewReadDataTrigger IDataViewSaveDataTrigger ITableReadDataTrigger ITableSaveDataTrigger Configurable The Admin Tool can be used to associate a Data Trigger with a dataview or a table. The trigger can also be disabled without being removed for testing and troubleshooting purposes Powerful A data trigger is not restricted to affecting only the database – it is full-fledged .NET code. It could send an email, manipulate an image, write to a file, generate a report, call a web service, etc. Granular Multiple data triggers can be defined for each table or dataview, meaning each data trigger can perform one specific task instead of having to perform multiple, possibly unrelated tasks. Also code within the data trigger itself can be written to do nothing unless the data meets specific criteria, so row-level granularity is possible. 841007837 Page |7 Extending the Schema The GRIN-Global schema is partially derived from GRIN, its predecessor. Much of the data tracked by genebanks world-wide is similar and is included in the base schema, but since most genebanks do things slightly differently, GRIN-Global makes it relatively easy to add new tables to the schema and have the GRIN-Global software manipulate its data – with no change to the software itself. The following three steps are all that are needed: 1. Create a new table definition in the database schema 2. Connect GRIN-Global with the new table 3. Indicate to GRIN-Global how to display the table’s data Create a Table Definition GRIN-Global supports multiple database engines. Since applying schema changes to a particular database engine varies greatly from another one, GRIN-Global does not provide a mechanism for creating the table definition itself. Consult the documentation of your particular database engine for details on how to accomplish this task. Connect GRIN-Global with the new Table Inspecting a database schema can take several seconds depending on which database engine GRINGlobal is using. For this reason, all tables must be explicitly “mapped” – cached in the GRIN-Global system tables themselves – before they can be used by GRIN-Global. This action needs only be done when schema changes have occurred. The steps to do this are: 1. Open the GRIN-Global Admin Tool. 2. Right-click Table Mappings, select Inspect Schema for Unmapped Tables. 3. Choose which table(s) to map. 4. Click Generate Default Mappings. Indicate to GRIN-Global How to Display the Table’s Data After the previous steps’ table mapping has been completed, a dataview can be created to read and write data to the table. To create a default dataview to see all the data in the new table: 1. Open the GRIN-Global Admin Tool. 2. Right-click Dataviews; select New Dataview. 3. Find the table in the treeview in the left panel; drag the table name to the textbox on the right. 4. Click the Properties tab and provide a unique name for the dataview. 5. Click the Titles tab and provide a language-specific title and description for the dataview. 6. Click Save. 841007837 Page |8 Creating a Data Trigger Usually a dataview will suffice for basic reading and writing of data. However, there may be instances when pre- or post-processing will be performed. For example, there may be a business rule that the curator should be emailed whenever inventory falls below a certain threshold. This functionality is handled by creating a custom data trigger. NOTE: Data triggers require the ability to write and compile .NET code. The tools for doing this are freely available from Microsoft. At the high level, creating a data trigger consists of: 1. Creating a .NET assembly (.dll file) 2. Associating the assembly with a particular dataview or resource in GRIN-Global Creating a .NET assembly The following basic steps are required to create a .NET assembly: 1. Open Visual Studio 2005 or greater 2. Create a new Class Library project 3. Add a reference to GrinGlobal.Interface.dll 4. Implement at least one Data Trigger interface (there are four defined in GrinGlobal.Interface.Dataviews.*) 5. Compile Associating a Data Trigger and a Resource Associating a data trigger to a particular resource is similar to the “table mappings” described earlier – GRIN-Global needs to be given the data trigger assembly and told when it should run the assembly’s code. This can be done very easily: 1. Open the GRIN-Global Admin Tool 2. Right click Data Triggers; select New Data Trigger 3. Click Add New… to find your assembly file 4. Choose the resource with which to associate the data trigger –a dataview, a table, or neither (meaning any resource) 5. Select which class contained in the assembly should be loaded for that resource 6. Click Save 841007837 Page |9