* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Introduction to Databases
Survey
Document related concepts
Transcript
Introduction to Databases Vetle I. Torvik DNA was the 20th century Databases are the 21st century Quantum leaps in the evolution of human brain power – Way-back-when: information in books - phone books, dictionaries, lab notebooks, journals – Recently: information at your fingertips – Now: scientific discovery at your fingertips • data mining bio-informatics databases • data mining text data bases How do you find a good movie? New releases only? Browsing shelves by category (comedy, action, drama, foreign, etc.)? Browsing through a book at blockbuster – – – – by titles alphabetically? by actors alphabetically? by category? by year? A step up... querying a database Now imagine this… Visualizing the entire movie database in ONE figure across ALL dimensions – year, category, actor, director, popularity, rating, length, language, country, awards, etc. and drilling down to find your movie(s) PS: You don’t have to imagine... Why not do the same in the scientific literature? Benefits of DBs Over paper books… a quantum leap – Speed, space, less drudgery Over spreadsheets … another quantum leap – Maintenance (less redundancy, etc) – Currency (accuracy, up-to-date, on-demand) – Access (across time and space, sharing) – Security (recovery, restrict others’ access) – Facilitates data mining: encode meaning, inferences, pooling/sharing, visualization A Database – an electronic repository for persistent data A Database system to store, retrieve, and manipulate data consists of 4 parts – Data - collection of linked data files – Hardware - for storage and execution – Software - DB management system (e.g., Access, MySQL, Filemaker, Oracle) – Users - DB administrator, data administrator, application programmers, end users Relational DBMSs Dominates market Data is perceived by users as tables only • representing, manipulating, and enforcing integrity of data so that operations function correctly • no duplicate records, rows and columns are unordered, each entry has a single value SQL = “structured query language” • a standard language for querying databases • independent of how the data is stored/accessed Database design - a subjective exercise Entity/Relationship diagramming – identify entities or “things that can be distinctly identified” • e.g. movie, category, individual(director, actor) – identify relationships • e.g. a movie has one director, zero or more actors, belongs to one category – draw the diagram Then “normalize” the database Ontologies - the basis upon which the truth of the world is viewed E.g. a movie has one director, zero or more actors, belongs to one category makes databases a bit more intelligent allows for making inferences – “the artist formerly known as Prince” - without an artist name, nobody can make any name related inferences about him… Metadata - data about the data It would be nice if SQL knew that actors and directors are both individuals so that (e.g.) querying movies by actor = director makes sense (and this type of query could be optimized) Data mining Searching for novel patterns, rules or relationships in data, e.g.: – – – – correlations classification clustering visualization Versus traditional statistics: hypothesis testing Data mining - correlations Searching through many possible pairs of associations to find novel ones, e.g.: – phenotypes versus genotypes Data mining - classification find rules that discriminate between predefined categories – e.g., breast cancer diagnosis – – – RULE #1: IF the following conditions hold ALL true at the SAME TIME, THEN the case is: "intra-ductal carcinoma” CONDITIONS: • The volume of the calcifications is more than 0.03 cm^3. • AND The total number of calcifications is greater than 10. • AND The variation in shape is moderate or marked. • AND The irregularity in size of calcifications is marked. • AND The variation of the density of calcifications is moderate or marked. • AND There is no ductal orientation. • AND The number of calcifications per cm^3 is less than 20. • AND A comparison with previous exams shows a change in the number or character of calcifications or it is newly developed. RULE #2: ... Data mining - clustering organizing information by naturally occurring groups, e.g.: – cluster languages by similarity of words to assess their evolution – organizing webpages into themes by word usage (e.g., www.vivisimo.com) – grouping genes by expression level in DNA microarrays to find a subset of differentially expressed genes Data mining - clustering Data mining - visualization Looking for patterns across multiple dimensions, and levels of resolution e.g.: – scientific collaboration behavior across time and subjects – map of power outage over time (what was the chain of events causing a major outage?) Data mining begins at home Your lab notebook is a database. Can you data mine your lab notebook?