Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
UC Berkeley Using Statistical Machine Learning in Cloud Computing ` David Patterson, UC Berkeley Reliable Adaptive Distributed Systems Lab 1 Image: John Curley http://www.flickr.com/photos/jay_que/1834540/ Datacenter is new “server” • • • • “Program” == Web search, email, map/GIS, … “Computer” == 1000’s computers, storage, network Warehouse-sized facilities and workloads New datacenter ideas (2007-2008): truck container (Sun), floating (Google), datacenter-in-a-tent (Microsoft) • How to enable innovation in new services without first building & capitalizing a large company? 2 photos: Sun Microsystems & datacenterknowledge.com Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 3 Cloud Computing is Hot • 9/15/09 Federal CIO Vivek Kundra embraces Cloud Computing • “We’ve been building data center after data center, acquiring application after application, and frankly what it’s done is drive up the cost of technology immensely across the board. What we need is to find a more innovative path in addressing these problems.” 4 But... What is cloud computing, exactly? 5 “It’s nothing (new)” “...we’ve redefined Cloud Computing to include everything that we already do... I don’t understand what we would do differently ... other than change the wording of some of our ads.” Larry Ellison, CEO, Oracle (Wall Street Journal, Sept. 26, 2008) 6 Above the Clouds: A Berkeley View of Cloud Computing abovetheclouds.cs.berkeley.edu • 2/09 White paper by RAD Lab PI’s and students – Clarify terminology around Cloud Computing – Quantify comparison with conventional computing – Identify Cloud Computing challenges & opportunities • Why can we offer new perspective? – Strong engagement with industry – Users of cloud computing in our own research and teaching in last 18 months • Goal: stimulate discussion on what’s really new – without resorting to weather analogies ad nauseam 7 Utility Computing Arrives • Amazon Elastic Compute Cloud (EC2) • “Compute unit” rental: $0.10-0.80/hr. – 1 CU ≈ 1.0-1.2 GHz 2007 AMD Opteron/Xeon core “Instances” Platform Cores Small - $0.10 / hr 32-bit 1 1.7 GB 160 GB Large - $0.40 / hr 64-bit 4 7.5 GB 850 GB – 2 spindles 64-bit 8 • N - $0.80 / hr XLarge Memory Disk 15.0 GB 1690 GB – 3 spindles • No up-front cost, no contract, no minimum • Billing rounded to nearest hour; pay-as-you-go storage also available • A new paradigm (!) for deploying services? 8 What is it? What’s new? • Old idea: Software as a Service (SaaS) – Basic idea predates MULTICS – Software hosted in the infrastructure vs. installed on local servers or desktops; dumb (but brawny) terminals • New: pay-as-you-go utility computing – Illusion of infinite resources on demand – Fine-grained billing: release == don’t pay – Earlier examples: Sun, Intel Computing Services—longer commitment, more $$$/hour, no storage – Public (utility) vs. private clouds 9 Why Now (not then)? • “The Web Space Race”: Build-out of extremely large datacenters (10,000’s of commodity PCs) – Build-out driven by growth in demand (more users) => Infrastructure software: e.g., Google File System => Operational expertise: failover, DDoS, firewalls... – Discovered economy of scale: 5-7x cheaper than provisioning a medium-sized (100’s machines) facility • More pervasive broadband Internet • Commoditization of HW & SW – Fast Virtualization – Standardized software stacks 10 Classifying Clouds • • • • Instruction Set VM (Amazon EC2) Managed runtime VM (Microsoft Azure) Framework VM (Google AppEngine) Tradeoff: flexibility/portability vs. “built in” functionality Lower-level, Less managed EC2 Higher-level, More managed Azure AppEngine 11 Cloud Economics 101 • Cloud Computing User: Static provisioning for peak - wasteful, but necessary for SLA Machines Capacity $ Capacity Demand Demand Time Time “Statically provisioned” data center “Virtual” data center in the cloud Unused resources 12 Cloud Economics 101 Machines Capacity Energy • Cloud Computing Provider: Could save energy Capacity Demand Demand Time Time “Statically provisioned” data center Real data center in the cloud Unused resources 13 Risk of Under Utilization • Underutilization results if “peak” predictions are too optimistic Capacity Resources Unused resources Demand Time Static data center 14 2 1 Time (days) Capacity Demand Capacity 2 1 Time (days) Demand Lost revenue 3 Resources Resources Resources Risks of Under Provisioning 3 Capacity Demand 2 1 Time (days) Lost users 3 15 New Scenarios Enabled by “Risk Transfer” to Cloud • “Cost associativity”: 1,000 CPUs for 1 hour same price as 1 CPUs for 1,000 hours (@$0.10/hour) – Washington Post converted Hillary Clinton’s travel documents to post on WWW <1 day after released – RAD Lab graduate students demonstrate improved Hadoop (batch job) scheduler—on 1,000 servers • Major enabler for SaaS startups – Animoto traffic doubled every 12 hours for 3 days when released as Facebook plug-in – Scaled from 50 to >3500 servers – ...then scaled back down 16 Cloud Computing & Statistical Machine Learning? • Before CC, only performance optimization on mostly small scale systems • CC detailed cost-performance model – Optimization more difficult with more metrics • CC Everyone can use 1000+ servers – Optimization more difficult at large scale • Economics rewards scale up AND down – Optimization more difficult if add/drop servers • SML as optimization difficulty increases 17 RAD Lab 5-year Mission Enable 1 person to develop, deploy, operate next -generation Internet application • Key enabling technology: Statistical machine learning – management, scaling, anomaly detection, performance prediction, ... • Highly interdisciplinary faculty & students – PI’s: Fox/Katz/Patterson (systems/networks), Jordan (machine learning), Stoica (networks & P2P), Joseph (systems/security), Franklin (databases) – 2 postdocs, ~30 PhD students, ~5 undergrads 18 RAD Lab Prototype: System Architecture Drivers Drivers Drivers Automatic Workload Evaluation (AWE) Director Offered load, resource utilization, etc. Training data performance & cost models Log Mining Chukwa & XTrace (monitoring) New apps, equipment, global policies (eg SLA) SCADS Chukwa trace coll. local OS functions Web 2.0 apps web svc Ruby on APIs Rails environment Chukwa trace coll. local OS functions VM monitor 19 Successes 1. Automatically add/drop servers to fit demand, without violating Service Level Agreement (SLA) 2. Predict performance of complex software system when demand is scaled up 3. Distill millions of lines of log messages into an operator-friendly “decision tree” that pinpoints “unusual” incidents/conditions • Recurring theme: cutting-edge Statistical Machine Learning (SML) works where simpler methods have failed 20 Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 21 Automatic Management of a Datacenter • As datacenters grow, need to automatically manage the applications and resources – examples: • deploy applications • change configuration, add/remove virtual machines • recover from failures • Director: – mechanism for executing datacenter actions • Advisors: – intelligence behind datacenter management 22 Director Framework workload model performance model Advisor Advisor Advisor Advisor monitoring data Director Drivers config Datacenter(s) VM VM VM VM 23 Director Framework • Director – issues low-level/physical actions to the DC/VMs • request a VM, start/stop a service – manage configuration of the datacenter • list of applications, VMs, … • Advisors – update performance, utilization metrics – use workload, performance models – issue logical actions to the Director • start an app, add 2 app servers 24 What About Storage? • Easy to imagine how to scale up and scale down computation • Database don’t scale down, usually run into limits when scaling up • What would it mean to have datacenter storage that could scale up and down as well so as to save money for storage in idle times? 25 DC Storage Motivation • Most popular websites follow the same pattern – Rapidly developed on SQLServer / PostgreSQL / MySQL / etc. – Become popular and realize scaling limitations – Build large, complicated ad-hoc systems to deal with scaling limitations as they arise • Websites that can’t scale fast enough lose customers SCADS: Scalable, ConsistencyAdjustable Data Storage • Scale Independence - as the user base grows: – No changes to application – Cost per user doesn’t increase – Request latency doesn’t change • Key Innovations 1.Performance safe query language 2.Declarative performance/consistency tradeoffs 3.Automatic scale up and down using machine learning Scale Independence Arch • Developers provide performance safe queries along with consistency requirements • Use ML, workload information, and requirements to provision proactively via repartitioning keys and replicas SCADS Performance Model (on m1.small, all data in memory) 5% writes 1% writes SLA threshold 99th percentile median Low workload, low put rate Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 31 Mining Console Logs for LargeScale System Problem Detection • Why console logs? • Console logs are everywhere – In almost every software system – Hand-picked information by developers – Expressive, convenient to use • Low programmer overhead • Especially useful in large scale Internet services – Open source code + in-house development – Continuously changing system “Detecting Large-Scale System Problem Detection by Mining Console Logs,” Wei Xu, Ling Huang, Armando Fox, David Patterson, Michael Jordan, 32 Symposium on Operating Systems Principles (SOSP), Oct 2009 Console logs use awkward languages HODIE NATUS EST RADICI FRATER* today unto the Root a brother is born. * "that crazy Multics error message in Latin." http://www.multicians.org/hodie-natus-est.html 33 Console logs are not operator friendly Console Logs Operators grep Perl scripts search • Problem – Don’t know what to look for! • Console logs are intended for a single developer • Assumption: log writer == log reader • Today many developers => massive textual logs • Our goal - Discover the most interesting log messages without any prior input 34 Console logs are hard for machines too Machine Parsing Learning Feature Creation Machine Learning Visualization • Problem • Highly unstructured, looks like free text • Not able to capture detailed program state with texts • Hard for operators to understand detection results • Our contribution • A general framework for processing console logs • Efficient parsing and features 35 Key observations • Parsing – Console logs were inherentedly structured • Determined by log printing statement – Constant strings = markers of message structure – Source code is usually available • Features • Common information reported in logs • Identifiers and execution paths • State variables • Correlations among messages • Many ways to group related log messages • i.e. not just by time 36 Case studies • Surveyed a number of software apps – Linux kernel, OpenSSH, Apache, MySQL, Jetty – Hadoop, Cassandra, Nutch … – Parsing works on 20/22 • In this talk – Hadoop file system – Per block operation logging (usually ignored) • Experiment on EC2 cloud – 203 nodes * 48 hours – ~ 300 TB HDFS data (550,000 blocks) – ~24 million lines of console logs 37 Step 1: Using source code analysis to help log parsing • Basic ideas 091012 03-00-00 INFO Creating file mydata Log.info(“Creating file ” + filename); Creating file (.*) [filename] • Non-trivial in object oriented languages – toString() method of objects – Dynamic binding (sub-classing) • Our Solution – Analyze source code to abstract syntax tree (AST) level – Pre-compute all possible message types • Free text -> semistructured text 38 Step 2: Feature Message count vector • Identifiers are common in logs – file names, object keys, user ids • Identifiers can be discovered automatically – Have many distinct values – Appear in multiple message types – Reported many times myfile: Creating Write • Grouping by identifiers – Similar to execution paths Backup Done • Numerical representation of these “paths” – Similar to Bag of words model in IR 39 Message count vector example datanode_r16 | Receiving block blk_100 src: … dest:... namenode_r10 | allocateBlock: blk_100 namenode_r10 | allocateBlock: blk_200 datanode_r16 | Receiving block blk_200 src: … dest:... datanode_r14 | Receiving block blk_100 src: …dest:… datanode_r16 | Received block blk_100 of size 49486737 from … datanode_r14 | Received block blk_100 of size 49486737 from … datanode_r16 | Error Receiving block blk_200 of size 49486737 from … blk_100 0 2 21200200000000 blk_200 0 1 01200201 000000 40 Step 3: Mining PCA detection 0221200201000000 55,000 vectors, one per block • This example - detect abnormal vectors • Dimensions highly correlated – Rare correlations indicate abnormal executions – PCA separates normal pattern from abnormal, making anomalies easy to detect • From text -> structured feature – With details of program execution – Can apply many other machine learning algorithms 41 PCA detection results Seq Event Description 1 2 3 4 5 6 7 8 9 10 11 Forgot to update namenode for deleted block Write block exception then client give up Failed at beginning, no block written Over-replicate-immediately-deleted Received block that does not belong to any file Redundant addStoredBlock request received Trying to delete a block, but the block no longer exists on data node Empty packet for block Exception in receiveBlock for block PendingReplicationMonitor timed out Other anomalies Total (Out of 550,000 blocks) Seq Description 1 2 False Positives Normal background migration Multiple replica ( for task / jobdesc files ) Total Actual Detected 4297 3225 2950 2809 1240 953 724 476 89 45 108 4297 3225 2950 2788 1228 953 650 476 89 45 107 16916 16808 False Positives 1397 349 1746 Making results easy for operators to understand! 42 Explaining detection results with decision tree writeBlock # received exception >=1 0 # Starting thread to transfer block # to # >=3 <=2 #: Got exception while serving # to #:# 0 >=1 1 0 addStoredBlock request received for # on # size # But it does not belong to any file >=1 1 0 # starting thread to transfer block # to # >=1 0 0 #Verification succeeded for # >=1 0 0 Receiving block # src: # dest: # >=3 0 1 >=1 0 Unexpected error trying to delete block #\. BlockInfo Not found in volumeMap 1 <=2 1 43 Summary: Mining Console Logs for Large-Scale System Problem Detection Extract Detect abnormal log segments 200 nodes, >24 million lines of logs Visualize A single decision tree to visualize system behavior Contributions: How to parse logs, How to turn into features so ready for SML, How to make results accessible to operators, Which SML algorithm used matters less 44 Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 45 Using SML to Characterize, Predict, Optimize Resources Given a system (e.g. Web Service, multicore …), its workload, and performance metrics: • Answer “what-if” questions for – Changes in workload (mix/rate/distribution) – Changes in system configuration (hw/sw) – Changes in both • 3 Examples for Case Study 3: – Parallel Database Performance Prediction – Hadoop Job Performance Prediction – Multi-core Performance Optimization (if time permits) Problem Statement Workload SYSTEM Behavior Model • Find relationships between workload and performance • How to generate the model? – Manually constructed queueing model – Automatically constructed • Regression, genetic algorithms, correlation analysis,… – This talk: Canonical Correlation Analysis Finding Correlations Kernel Canonical Correlation Analysis (KCCA) • 2003, Bach & Jordan at Berkeley • Find dimensions of maximal correlation between kernelized datasets – “Kernel functions” impose system-specific definition of similarity • Dimensionality reduction + clustering effect Advantages: – Relative distance between datapoints (defines neighborhoods) – Interpolation (preserves neighborhoods across both datasets) 3 Challenges in using SML 1. Mapping function from traces to feature vectors • • Workload -> multidimensional space System behavior -> multidimensional space 2. How to compare similarity of two feature vectors 3. How to leverage results of SML techniques to find actionable answer Hadoop Job Performance Prediction Jobs Hadoop Performance 51 Motivating Scenarios • Which jobs should I schedule together to avoid resource contention in Hadoop? – Should I run my job now or wait till later? • What is the optimal number of nodes to run a job on? – How many nodes should my job run on to meet the deadline? • Given observed behavior of a job run at small scale, how will the job behave when scaled up? – Are the performance bottlenecks the same? – Which aspects of the system perform non-linearly? 52 Hadoop Facts • Map-reduce jobs have diverse performance profiles - Each job has a different behavioral signature - Input parameters affect scaling slopes of jobs • Map performance is linear wrt number of nodes • Reduce complexity is job specific • Shuffle step is primary source of job unpredictability 53 SML Goals • Predict runtime/resource requirements for jobs – Same jobs, different configs – Same configs, different jobs • Identify which jobs can be co-scheduled – minimize resource contention – maximize shared data • Create Map-Reduce benchmark – Representative workloads – Varying scale factors 54 Data and Testbed: Facebook • Multi-user environment • Data set spans 6 months • Know job names but not job source code • ~600 node cluster • Nodes 1-300 – 16GB memory – 5 map slots – 5 reduce slots • Nodes 301-600 – 8GB memory – 5 map slots – 0 reduce slots Recurring Extract Transform Load (ETL) Jobs # of instances 66 unique jobs account for 99% of ETL jobs JobID Most repeated job 14000 12000 Total Time 10000 8000 6000 4000 2000 0 Same job, many configs 14000 12000 Total Time 10000 8000 6000 4000 2000 0 0 50 100 150 Number of Maps 200 250 X Y Job Descriptor Vector Hadoop • Job config parameters • Number of Maps • Number of Reduces • Data characteristics • Map Input Bytes • Locality Performance Vector • Map Time • Reduce Time • Total Time • Map Output Bytes Predicting Job Performance Actual time (ms) Total Time Predicted time (ms) Actual bytes Map Output Bytes Predicted bytes Multi-Core Performance Optimization (if time permits) Application Optimization Configurations Multi-core Performance A. Ganapathi, K. Datta, A. Fox, D. Patterson, "A Case for Machine Learning to Optimize Multicore Performance", First USENIX Workshop on Hot Topics in Parallelism (HotPar '09), Berkeley, CA, March 30-31, 2009. Problem Structured Grids Dense Linear Algebra Motifs + Sparse Linear Algebra Diverse Sun Intel/AMD IBM Multicore Niagara2 x86 Blue Gene Architectures + Compilers alonexlc icc (No code tuning) gcc = Poor Performance! Solution: Auto-tuning Identify motif-specific optimizations Generate code variants based on these optimizations Benefits: Problem: • Portable across diverse architectures Optimization • Easily scalable Thread Count • Tunable for any relevant metric Domain Decomposition • Minimizes programmer tuning Software Prefetching time • Proven track record (e.g. ATLAS, Padding SPIRAL, FFTW, OSKI) Inner Loop • Already showed speedups of Total 1.5x to 5.6x Search parameter space for best configuration Parameters Total Configs 1 4 4 36 2 18 1 32 8 480 16 4x107 ML Goals Goal: find best value for configuration parameters to simultaneously optimize multiple performance metrics Require little architecture/application knowledge Provide actionable suggestions for improving performance Simultaneously optimize for multiple performance metrics (e.g. energy efficiency and cycle time) Use small set of samples to build model Experimental Setup 2563 regular grid x+1 z+1 y+1 y-1 (x,y,z) z-1 x-1 Structured Grids (stencil codes) • • 3D 7-point stencil • • 3D 27-point stencil Used in iterative PDE solvers for image processing, diffusion etc. For a given point, a stencil is a fixed subset of nearest neighbors A stencil code updates every point in a regular grid by “applying a stencil” Stencil codes usually bandwidthbound Architectures • AMD Barcelona 2 sockets x 4 cores/socket, 2.30 GHz, compiler: gcc • Intel Clovertown 2 sockets x 4 cores/socket, 2.66 GHz, compiler: icc Potential Optimizations TY +X NX(unit stride) CY Thread 0 CZ RXRY RZ CX Domain Decomposition Processing … A[i-1] A[i] A[i+1] Low Memory Bandwidth Retrieving from DRAM … A[i+dist-1] A[i+dist] A[i+dist+1] Thread 1 Conflict Misses … NY TY CZ NZ +Z +Y TX Parallelization and Capacity Misses TX Software Prefetching Thread n Array Padding for i=1 to x { for i=1 to xto { y{ for j=1 for j=1 to { z{ for i=1 to { y to forxk=1 for k=1 for j=1 to y to { z{ A[i,j,k]=… A[i,j,k]=… for k=1 } to z { }} A[i,j,k]=… }}} }} } Poor Functional Unit Usage and Scheduling Inner Loop Opts. 3 Challenges in using ML 1. Mapping function from traces to feature vectors • • Configuration parameters -> multidimensional space System behavior -> multidimensional space 2. How to compare similarity of two feature vectors 3. How to leverage results of ML techniques to find actionable answer Configuration Parameters Parameter Optimization Type Thread Count Number of Threads Block Size Domain Decomposition Parameter Range Number of Configurations NThreads {20…23} 4 CX {27…NX} CY {21…NY} CZ NZ Name 36 Chunk Size Software Prefetching Prefetching Type Padding Padding Size {register block, plane, pencil} 3 {0, 25…29} 6 {0…31} 32 Prefetching Distance Register Block Size Inner Loop Optimizations Feature Vector RX {20…21} RY {20…21} RZ {20…23} 10 Statement Type {complete, individual} 2 Read From Type {array, variable} 2 Pointer Type {fixed, moving} 2 Neighbor Index Type {register block, plane, pencil} 3 FMA-like Instructions {yes, no} 2 X Threads Block Size CX Block Size CY Block Size CZ Padding Size Prefetch type Prefetch distance Statement Type 4 32 128 256 32 Plane 64 individual Performance Metrics Counter Description PAPI_TOT_CYC Cycles per thread per job PAPI_L1_DCM PAPI_L2_DCM L1 data cache misses per thread L2 data cache misses per thread PAPI_TLB_DCM PAPI_CA_SHR TLB misses per thread Accesses to shared cache lines PAPI_CA_CLN PAPI_CA_ITV Accesses to clean cache lines Cache interventions Power meter Watts consumed per second Feature Vector Y (total cycles x # of flops) / (clk rate x # of watts) Total Cycles L1_DCM L2_DCM TLB_DCM CA_SHR CA_CLN CA_ITV Energy Efficiency 1.9E7 2.4E5 1.5E5 1.2E4 1.2E5 1.4E4 1.2E3 2.3E4 Kernel functions Threads Block Size CX Block Size CY Block Size CZ Padding Size Prefetch type Prefetch distance Statement Type 4 32 128 256 32 Plane 64 Individual 2 64 128 256 32 Plane 64 Individual X1 X2 2 64 128 256 32 Pencil 64 complete X3 Are X1 and X2 more similar than X2 and X3? Measure of similarity Numeric Columns: Gaussian Kernel K(xi,xj) = exp(-||xi-xj||2 /) Non-numeric Columns: K(xi,xj) = {1 if xi=xj, 0 if xi≠xj } Similarity(X1, X2) = average(K(X1i, X2i)) Stencil Code X Y Configuration Feature Vector Performance Feature Vector Platform kernel function kernel function • performance • compiler optimizations 1 … N counters 1 …N 1 1 • parameters for memory blocking • power meter readings : : KY : • Ksoftware prefetching : X N N KCCA 0 K K Kernel functions X Y A KXKX 0 = A K K 0 Gaussian B B • Numeric Columns: 0 K Kernel K • Non-numeric Columns: K(xi,xj) = {1 if xi=xj, 0KifY*Bxi≠xj } KX*A • Similarity(X1, X2) = average(K(X1i, X2i)) Y X Y Y Finding Optimal Configurations KCCA Data Space Raw Data Space Configuration features Performance Metrics X Y Inverse Image ??? Problem KX*A KCCA Nearest Neighbors KY*B Results 75 Auto-tuning Time • Exhaustive Search: – 4x107 configs x 0.08 s/trial x 5 trials > 180 days • Our Technique: – Training data: 1500 configs x 0.08 s/trial x 5 trials = 10 minutes – Training time using KCCA: 90 minutes – Genetic algorithm: 243 configs x 0.08 s/trial x 5 trials = 1.6 min – Total Time < 2 hrs 76 Summary: SML Strengths 1. Train vs. write the logic (handle SW churn) 2. Learns how to handle and make policy during transitions between steady states (unlike queueing theory) 3. Coping with a complex cost function (unlike simple control theory) 4. Finding trends, needles in data haystack 5. Fast enough to run online (today) – Infeasible algorithms from 1960s fast in 2008 – Computers cheap enough OK to “just” monitor 77 Summary: SML Accommodations • System data usually text vs. numbers – Need to turn text into features – Need to find distance function for close vs. far • SML is not perfect – System must live with <100% accuracy, false positives • Will be asked “Yes, I see that SML works, but why didn’t you try ‘simpler’ Algorithm X?” – For all X – Simpler = I know X, I don’t know SML? 78 Conclusion • Cloud Computing will transform IT industry – Pay-as-you-go utility computing leveraging economies of scale of Cloud provider – Anyone can create/scale next eBay, Twitter… • Cloud Computing better management: cost-performance, scale, scale up/down • SML: making sense from lots of data • RAD Lab applying SML to New Cloud Computing challenges: Director, SCADS, Map Reduce Scheduling, Log Analysys 79 Acknowledgments • Peter Bodik for Director/SCADS experiments • Wei Xu for Log Analysis • Archana Ganapathi for Map Reduce and Multicore Performance Prediction • 30 PhD students and 8 faculty of RAD Lab • RAD Lab founding sponsors Google, Microsoft, and Sun Microsystems and affiliates Amazon Web Services, Cloudera, Cisco, Facebook, HP, NetApp, SAP, VMware 80 Backup Slides 81 Evaluation Benchmarks – ordered the optimizations – applied them consecutively – more information: Datta et al., “Stencil Computation Optimization and Auto-tuning on State-of-the-Art Multicore Architectures”, Supercomputing 2008. Opt. #2 Parameters • “No Optimization”: running naïve code • “Expert Optimized”: Opt. #1 Parameters • “Random Raw Data”: best performing point in raw data • “Genetic on Raw Data”: permute configs for top three best performing points in raw data Cross Validation Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Test Set Map Time Reduce Time Total Time Map Output Bytes mod4 (old) 0.60 0.80 0.70 0.50 mod3 0.84 0.85 0.86 0.94 mod2 0.81 0.76 0.83 0.94 mod1 0.91 0.77 0.88 0.97 mod0 (new) 0.96 0.89 0.97 0.91 Train with Sampled Subset Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Training Set Map Time Reduce Time Total Time Map Output Bytes All old data 0.96 0.89 0.97 0.91 sample1 0.93 0.87 0.94 0.85 sample2 0.62 0.94 0.81 0.81 sample3 0.76 0.91 0.91 0.72 sample4 0.64 0.83 0.78 0.71 sample5 0.69 0.87 0.82 0.74 Microsoft’s Chicago Modular Datacenter 85 The Million Server Datacenter • 24000 sq. m housing 400 containers – Each container contains 2500 servers – Integrated computing, networking, power, cooling systems • 300 MW supplied from two power substations situated on opposite sides of the datacenter • Dual water-based cooling systems circulate cold water to containers, eliminating need for air conditioned rooms86 High workload, low put rate High workload, high put rate Low workload, low put rate 2020 IT Carbon Footprint 820m tons CO2 360m tons CO2 2007 Worldwide IT carbon footprint: 2% = 830 m tons CO2 Comparable to the global aviation industry Expected to grow to 4% by 2020 260m tons CO2 90 Energy & Cloud Computing? • Cloud Computing saves Energy? • Don’t buy machines for local use that are often idle • Better to ship bits as photons over fiber vs. ship electrons over transmission lines to spin disks, power processors locally – Clouds use nearby (hydroelectric) power – Leverage economies of scale of cooling, power distribution 91 Energy & Cloud Computing? • Techniques developed to stop using idle servers to save money in Cloud Computing can also be used to save power – Up to Cloud Computing Provider to decide what to do with idle resources • New Requirement: Scale DOWN and up – Who decides when to scale down in a datacenter? – How can Datacenter storage systems improve energy? 92 Hybrid / Surge Computing • Keep a local “private cloud” running same protocols as public cloud • When need more, “surge” onto public cloud, and scale back when need fulfilled • Saves energy (and capital expenditures) by not buying and deploying power distribution, cooling, machines that are mostly idle 93