Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Module 8 ********* Data Validation & Security Workforce Information Database Training Last update November 2006 Module 8 Data Validation & Security Providing Accurate Data Consistently Data validation allows us to provide data with confidence in its accuracy, and we can consistently provide this data by implementing thorough security. Module 8 Data Validation & Security Data Integrity Validity, consistency, and accuracy of the data in a database. •Table-level •Field-level •Relationship-level •Business Rules Module 8 Data Validation & Security Data Validation The process of determining if an update to a value in a table’s data cell is within a preestablished range or is a member of a set of allowable values. Module 8 Data Validation & Security What are some common data quality problems that affect data integrity and validation? Module 8 Data Validation & Security Common Data Quality Problems • • • • • • • • • Illegal values Violated attribute dependencies Uniqueness violation Referential integrity violation Missing values Misspellings Cryptic values Embedded values Misfielded values Module 8 Data Validation & Security Common Data Quality Problems continued…. • • • • • • • • Word transpositions Duplicate records Contradicting records Wrong references Overlapping data/matching records Name conflicts Structural conflicts Inconsistencies Module 8 Data Validation & Security What are some methods for checking for data validity? Module 8 Data Validation & Security Methods for checking for data validity • • • • • • • Visual/manual Aggregation Reviewers guide Auto data checks Record counts Spell checks Have data provider review Module 8 Data Validation & Security Time to Exercise!! Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Security Concerns: Confidentiality Integrity Availability Web security Application security •Production vs. test User level Physical security (including backup & restoration plan) •Network permissions Data Storage Access (Server or PC) User level •Suppression flags •Data aggregation issues •RDBMS •ODBC Network (LAN/WAN) security Application security •Production vs. test User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Security Concerns: Confidentiality Integrity Availability Web security Application security •Production vs. test User level Physical security (including backup & restoration plan) •Network permissions Data Storage Access (Server or PC) User level •Suppression flags •Data aggregation issues •RDBMS •ODBC Network (LAN/WAN) security Application security •Production vs. test User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Module 8 Data Validation & Security Physical Security Questions to ask yourself: • Where is your data actually stored? • Are you responsible for physical security? If you are... Module 8 Data Validation & Security Physical Security • What physical security measures do you have in place? • Do you have the right hardware? • Are you using obsolete hardware that is prone to crashing or hacking? Do you have a choice? Module 8 Data Validation & Security Physical Security • Do you have a database backup plan? • Do you have remote backup so if a fire burns the building you don’t lose both your primary and backup data at the same time? • Do you have a data restoration plan? Does the backup plan allow for feasible restoration? Module 8 Data Validation & Security Physical Security Restoration Plan • Does your restoration plan allow for the restoration of individual tables or require the entire database? • Do you maintain copies of the tables on your hard drive? Module 8 Data Validation & Security Physical Security • Who has access to the physical storage location? • Is your database on a PC or server that someone could easily kick, spill coffee on, or accidentally reappropriate? If you are not responsible for physical security, ask these questions of those who are! Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Security Concerns: Confidentiality Integrity Availability Web security Application security •Production vs. test User level Physical security (including backup & restoration plan) •Network permissions Data Storage Access (Server or PC) User level •Suppression flags •Data aggregation issues •RDBMS •ODBC Network (LAN/WAN) security Application security •Production vs. test User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Module 8 Data Validation & Security Data Storage Access • What software are you using to store the database? • SQL Server, Oracle, FoxPro, Access? Survey says: SQL Server 48% Oracle 28% Other 24% Module 8 Data Validation & Security Data Storage Access • What can be done at the server level to provide for security? • Confidentiality, Integrity, Availability DO apply advanced security to the most sensitive data, but DON’T apply advanced security to non-sensitive data Module 8 Data Validation & Security Data Storage Access Security at the server and/or database level… • Permissions • What kind of permissions exist? • Read only, SA, etc. • Who sets those permissions? • Who has those permissions? Module 8 Data Validation & Security ALMIS Database Security Security at the data level… • Suppression • Suppression flags • Suppression can be handled at the database level or the application level Module 8 Data Validation & Security ALMIS Database Data Security Two approaches: 1. ALMIS Database contains NO confidential data (all data available for use without restriction) 2. ALMIS Database contains confidential data (data access controlled by security and/or suppression) Module 8 Data Validation & Security ALMIS Database Security Tables that have suppression flags... ces iomatrix stindprj indprj occprj stoccprj industry oeswage NOTE: tables stfirms and wage have no suppression flags but may contain confidential data Module 8 Data Validation & Security ALMIS Database Data Security Issues to consider regarding suppression: • If your database doesn’t contain suppressed records, detailed data won’t aggregate to totals (validity checks?) • Without proper suppression, confidential data can be back-calculated Module 8 Data Validation & Security ALMIS Database Data Security A note on confidential data: If you are checking 202 data (or any other potentially confidential data) to see that it loaded right, make sure your EQUI data files and/or printouts are always secured and shred all photocopies when finished. Check with your local BLS personnel or LMI administrator for specific confidentiality policies. Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Security Concerns: Confidentiality Integrity Availability Web security Application security •Production vs. test User level Physical security (including backup & restoration plan) •Network permissions Data Storage Access (Server or PC) User level •Suppression flags •Data aggregation issues •RDBMS •ODBC Network (LAN/WAN) security Application security •Production vs. test User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Module 8 Data Validation & Security User Access Three major types of user access: • PC - direct to database (ODBC or RDBMS) • Network (LAN/WAN) through application • Web (passive or active through application) Module 8 Data Validation & Security User Access via the Web • How do Web users access the data from the database? • Passive (static tables automatically updated to web pages) • Active (query through application) Module 8 Data Validation & Security User Access via the Web Questions to ask yourself: • What kind of web server are you using? What are the security considerations with that choice? • What kind of firewall do you have? Module 8 Data Validation & Security User Access via the Web • Does your web interface have security/confidentiality suppression? • Do you display suppressed data to certain users? If so, how is access managed? Passwords? IP address? Module 8 Data Validation & Security User Access via a Network (LAN/WAN) Questions to ask yourself: • What type of network system do you have? • Who has rights/access to your data via the network? Through what applications? • Are confidential data (suppressed records) available? Module 8 Data Validation & Security User Access via a Network (LAN/WAN) • How do you control access to confidential data? Through user Ids? Through server access permissions? • DO recognize the important distinction between network security and data security. Module 8 Data Validation & Security User Access via direct connection Questions to ask yourself: • Is local access machine password protected? • Who has access to your machine? • Is there a backup plan for your access machine? • Is the source data for your database secure? Backup plan? Restorable? Module 8 Data Validation & Security ALMIS Database Data Security Documentation It may prove beneficial to keep detailed records on… • • • • • How security is done Where security exists Who is responsible for security Who has access/permissions to what Etc... Module 8 Data Validation & Security Data Security DISCUSSION