Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Extensible Storage Engine wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Relational model wikipedia , lookup
Functional Database Model wikipedia , lookup
Clusterpoint wikipedia , lookup
ORACLE DATA SHEET Oracle Big Data SQL Release 3.1 The unprecedented explosion in data that can be made useful to enterprises – from the Internet of Things, to the social streams of global customer bases – has created a tremendous opportunity for businesses. However, with the enormous possibilities of Big Data, there can also be enormous complexity. Integrating Big Data systems to leverage these vast new data resources with existing information estates can be challenging. Valuable data may be stored in a system separate from where the majority of business-critical operations take place. Moreover, accessing this data may require significant investment in re-developing code for analysis and reporting - delaying access to data as well as reducing the ultimate value of the data to the business. Oracle Big Data SQL enables organizations to immediately analyze data across Apache Hadoop, NoSQL and Oracle Database leveraging their existing SQL skills, security policies and applications with extreme performance. From simplifying data science efforts to unlocking data lakes, Big Data SQL makes the benefits of Big Data available to the largest group of end users possible. KEY FEATURES • • • Seamlessly query data across Oracle Database, Hadoop and NoSQL sources Runs all Oracle SQL queries without modification – preserving application investment Smart Scan on Hadoop, NoSQL and Oracle tablespaces enhance performance by parsing and intelligently filtering data where it resides • Enables Oracle Database 12c access to leading Hadoop distributions • Oracle Database Security features provide single access control to sensitive data across Oracle Database, Hadoop and NoSQL data • Easily copy data from Oracle Database to Hadoop using Copy to Rich SQL Processing on All Data Oracle Big Data SQL is a data virtualization innovation from Oracle. It is a new architecture and solution for SQL and other data APIs (such as REST and Node.js) on disparate data sets, seamlessly integrating data in Apache Hadoop and a number of NoSQL databases with data stored in Oracle Database. Using Oracle Big Data SQL, organizations can: • Combine data from Oracle Database, Apache Hadoop and NoSQL in a single SQL query • Query and analyze data in Apache Hadoop and NoSQL • Maximize query performance on all data using advanced techniques like Smart Scan, Partition Pruning, Storage Indexes, Bloom Filters and Predicate Push-Down in a distributed architecture • Integrate big data analyses into existing applications and architectures • Extend security and access policies from Oracle Database to data in Apache Hadoop and NoSQL ORACL E DA TA SH EET Hadoop Enhanced External Tables When dealing with large data sets stored in disparate systems, it can be difficult to know KEY BENEFITS • Transparently analyze data sets across Hadoop, NoSQL and Oracle Database • Achieve fast query performance by leveraging data local processing • Use your existing SQL skills to analyze data across big data sources • Current SQL-based applications can seamlessly integrate new data • • where your data is, let alone understand how the data is structured. Big Data SQL adds new external table types to Oracle Database 12c, which give users a single location to catalog and secure data in Hadoop and NoSQL systems: the Oracle Database. Big Data SQL keeps track of the metadata about external data sources – both clusters and the tables within them – without moving or copying data. External tables for Big Data SQL provide: • Seamless metadata integration and queries which join data from Oracle Database with data from Hadoop and NoSQL databases • Seamlessly extend Information Lifecycle Management strategy to leverage lower cost Hadoop storage Automatic mappings from metadata stored in HCatalog (or the Hive Metastore) to Oracle Tables • Multiple cluster support to allow one Oracle Database to query multiple Hadoop clusters Use Oracle Database security policies to keep all sensitive data safe • Enhanced access parameters to give database administrators the flexibility to control column mapping and data access behavior Figure 1. Oracle Big Data SQL enables Oracle SQL queries to span Oracle Database, Apache Hadoop and selected NoSQL data stores. Smart Scan: Data-Driven Parallel Processing Finding insights from Big Data can mean sifting through an extraordinary amount of data. With the massive increase in data volumes that Big Data brings, analytical performance can only be achieved by moving the analytics to the data, not the other way around. Big Data SQL applies the power of Smart Scan, first introduced in Oracle’s best-in-class Exadata Database Machine, to Hadoop and NoSQL systems. Smart Scan enables Oracle SQL operations to be pushed down to the storage tiers of the Big Data system. Paired with the horizontal scalability of these storage systems, Smart Scan automatically provides parallel processing equal to your biggest data set, enabling: • 2 | ORACLE BIG DATA SQL DATA SHEET Locally filtered data, so that only the rows and columns relevant to your query are transmitted to Oracle Database ORACL E DA TA SH EET • Join optimization via Bloom filters, speeding up joins between data in Oracle Database and massive data in Hadoop • Scoring for data mining models and enhanced processing for querying document data sets in for example JSON or XML • Oracle-native operators providing complete fidelity between queries run with Big Data SQL and Oracle Database alone Storage Indexing: More Effective I/O In addition to the set of Smart Scan features, Oracle Big Data SQL provides Storage Index technology to speed up processing before any I/O occurs. As data is accessed, Oracle Big Data SQL automatically builds local, in-memory indexes that capture where relevant data is stored. On subsequent queries of the same data, Storage Index technology ensures that data blocks that are not relevant to the query are not read. Because data blocks in Big Data systems can be very large (up to hundreds of megabytes), this “I/O skipping” strategy can improve performance on some queries by orders of magnitude. Predicate Push-Down: Harness External Storage Systems Oracle Big Data SQL not only enables easy integration of data from Hadoop and NoSQL sources, Big Data SQL also leverages the underlying storage mechanisms to provide the best possible performance. Big Data SQL’s predicate push down technology allows predicates in queries issued in Oracle Database to be executed by remote systems, and to be pushed into certain file formats. Using predicate push down, Big Data SQL enables you to: • Prune partitions from tables managed by Apache Hive • Minimize I/O on files stored in Apache Parquet and Apache ORC formats • Enable remote reads on data stored in Oracle NoSQL Database or Apache HBase Extend Information Lifecycle Management to Hadoop For many years, Oracle Database has provided rich support for Information Lifecycle Management (ILM). Numerous capabilities are available for data tiering – or storing data in different media based on access requirements and storage cost considerations. These tiers may scale from 1) in-memory for real time data analysis, 2) Database Flash for frequently accessed data, 3) Database Storage and Exadata Cells for queries of operational data and 4) Hadoop for infrequently accessed raw and archive data: 3 | ORACLE BIG DATA SQL DATA SHEET ORACL E DA TA SH EET Copy to Hadoop Copying data from Oracle Database to Hadoop can be complicated. Oracle Big Data SQL includes the Oracle Copy to Hadoop utility. This utility simplifies copying Oracle data to the Hadoop Distributed File System (HDFS). Data copied to the Hadoop cluster by Copy to Hadoop is stored in Oracle Data Pump format. This format optimizes queries thru Big Data SQL: 1) the data is stored as Oracle data types – eliminating data type conversions and 2) the data is queried directly – without requiring the overhead associated with Java SerDes. Native Hadoop tools like Hive can easily access these same Oracle Data Pump export files using optimized input format classes. Smart Scan on Tablespaces in HDFS Oracle Partitioning is the enabling technology that allows a single table’s data partitions to be stored on the various tiers. This enables immutable archive data within a table to reside in Hadoop in highly optimized Oracle Database storage formats. Database queries seamlessly access this archive data as they would any other data – exploiting the optimized access and storage structures (like indexes) for fast query performance. In addition, Big Data SQL’s Smart Scan capabilities enable compound performance benefits. Big Data SQL Smart Scan utilizes the massively parallel processing power of the Hadoop cluster to filter data at its source – greatly reducing data movement and network traffic between the cluster and the database. Oracle Database Security on Big Data Oracle Big Data SQL’s unique approach to integrating data enables Oracle Database Security features on Hadoop and NoSQL data. Using standard Oracle security mechanisms, you can secure Big Data using: • Standard Oracle Database roles and privileges to govern access to data • Data redaction, to ensure that sensitive information is obscured when accessed by unauthorized users • Virtual Private Databases to better enforce governance policies Supports a Range of Big Data Deployments Oracle Big Data SQL is designed to support a wide range of deployment options and platforms. Big Data SQL requires 1) any Oracle Database 12c (version 12.1.0.2 or higher) running Enterprise Linux and 2) leading Apache Hadoop distributions from Cloudera and Hortonworks. Big Data SQL achieves highest performance when paired with Oracle Engineered Systems. Big Data SQL takes full advantage of the power of Oracle Exadata and Oracle Big Data Appliance to create a best-in-class Big Data Management System, unifying the power of big data and Oracle Database. For those looking to bring the power of Oracle SQL to big data in the cloud, Big Data SQL Cloud Service will be available in combination with Oracle Exadata Cloud Service and Oracle Big Data Cloud Service. Oracle Database 4 | ORACLE BIG DATA SQL DATA SHEET Database Hardware Hadoop Cluster Hardware Hadoop Distribution and ORACL E DA TA SH EET Version 12.1.0.2 or higher 12.1.0.2 or higher Version Oracle Exadata (Linux OL6) or Any Intel x86 64-bit system (Linux OL6, RHEL6) Any Intel x86 64-bit system (Linux OL6, RHEL6) • CDH* 5.5 and higher Oracle Exadata (Linux OL6) or Any Intel x86 64-bit system (Linux OL6, RHEL6) Oracle Big Data Appliance (Linux OL5 and OL6) CDH 5.5 and higher • HDP ** 2.3 and higher * CDH: Cloudera’s Distribution Including Apache Hadoop ** HDP: Hortonworks Data Platform Getting Started Try using Oracle Big Data SQL – as well as other components of Oracle’s big data platform – in Oracle Big Data Lite Virtual Machine (http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite2104726.html). Big Data Lite allows you to test drive Oracle’s big data capabilities from your laptop or desktop computer. Capabilities include CDH, Oracle Big Data Spatial and Graph, Oracle Big Data Discovery, Oracle Big Data Connectors, Oracle Data Integrator, Oracle Golden Gate and more. CONTACT US For more information about Oracle Big Data SQL, visit oracle.com or call +1.800.ORACLE1 to speak to an Oracle representative. CONNECT WITH US blogs.oracle.com/oracle facebook.com/oracle twitter.com/oracle oracle.com Copyright © 2016, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0116 5 | ORACLE BIG DATA SQL DATA SHEET