Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
PLATFORM FOR BIG DATA, NOSQL AND RELATIONAL DATA. WHAT MAKES SENSE FOR ME? (+AZURE) [email protected] http://blogs.gartner.com/douglaney/files/2012/01/ad949-3D-Data-Management-ControllingData-Volume-Velocity-and-Variety.pdf RoadDesignator DrivingStatus A1 Difficulties Batch Processing Interactive Analysis Stream Processing Query runtime Minutes to hours Milliseconds to minutes Never-ending Data volume TBs to PBs GBs to PBs Continuous stream Programming model MapReduce Queries DAG Users Developers Analysts and developers Developers Originating project Google MapReduce Google Dremel Twitter Storm Open source project Hadoop / Spark Drill / Shark / Impala Hbase Storm / Apache S4 /Kafka What’s the social sentiment for my brand or products How do I better predict future outcomes? How do I optimize my fleet based on weather and traffic patterns? GAIN COMPETITIVE ADVANTAGE BY MOVING FIRST AND FAST IN YOUR INDUSTRY IT infrastructure optimization Churn analysis Fraud detection Life sciences research Legal discovery Social network analysis Traffic flow optimization Natural resource exploration Weather forecasting Healthcare outcomes Advertising analysis Equipment monitoring Web app optimization Smart meter monitoring persistent | distributed • In Memory • Efficient at Random Reads/Writes • Distributed, large scale data store • Utilizes Hadoop for persistence • Both HBase and Hadoop are distributed http://developer.yahoo.com/hadoop/tutorial/module4.html MANAGE ANY DATA, ANY SIZE, ANYWHERE 010101010101010101 1010101010101010 01010101010101 101010101010 Hadoop architecture. Distributed Processing (Map Reduce) Distributed Storage (HDFS) INSIGHTS FOR ALL USERS THROUGH FAMILIAR TOOLS PB TB GB An object contained within a user database Defines the scheme for the federation Represent the database being sharded Orders_federation Orders_federation Database that houses the federation object System managed SQL databases Contain part, or “slices” of data CREATE FEDERATION fed_name(fed_key_label fed_key_type distribution_type) The key used for data distribution int, bigint, guid, varbinary Represent a single instance of a federation key. All rows in all federated tables with the same federation key value. Orders_federation Orders_federation PK=5 PK=5 PK=25 PK=25 PK=35 PK=35 Contains only atomic units for member’s key range Non-Federated table SPLIT members to spread workloads over to more nodes DROP members to shrink back to fewer nodes ALTER FEDERATION Orders_Fed SPLIT AT (tenant_id=7500) SalesDB Orders_federation Orders_federation Orders_Fed [5000, 7500)[5000, & [7500,10000) 10000) Ensure apps can discover where the data is just-in-time No “Shard Map” caching Guaranteed member routing USE FEDERATION Orders_Fed (tenant_id=7509) SalesDB Orders_federation Orders_federation Orders_Fed [5000, 7500) & [7500, 10000) Table Details MANAGE ANY DATA, ANY SIZE ANYWHERE 1010101010101010 01010101010101 101010101010 Hadoop Connectors & ETL Frameworks Services Fabric Infrastructure .. . virtual machines .. . web sites .. . cloud services Automated Elastic Managed Resources Usage Based N Central US, S Central US, N Europe, W Europe, E Asia, SE Asia + 24 Edge CDN Locations SQL database .. . noSQL database .. . blob storage connect .. . virtual network traffic manager