Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
nanoHUB.org online simulations and more IEEE International Conference on Autonomic Computing (ICAC’06) Autonomic Live Adaptation of Virtual Computational Environments in a Multi-Domain Infrastructure Paul Ruth, Junghwan Rhee, Dongyan Xu Department of Computer Science and Center for Education and Research in Information Assurance and Security (CERIAS) Rick Kennell, Sebastien Goasguen Rosen Center for Advanced Computing Purdue University West Lafayette, Indiana, USA 1 Network for Computational Nanotechnology nanoHUB.org Outline of Talk online simulations and more • • • • • • • Motivations Overall architecture Design and implementation Real-world deployment in nanoHUB Related work Conclusion Demo 2 Network for Computational Nanotechnology nanoHUB.org Motivations online simulations and more • Formation of shared distributed cyberinfrastructure (CI) Spanning multiple domains Serving users/user communities with diverse computation needs Exhibiting dynamic resource availability and workload • Need for virtual distributed environments (VIOLINs), each with Customizability and legacy application compatibility Administrative privileges Isolation, security, and accountability Autonomic adaptation capability - A unique opportunity brought by virtualization (VMs and VNs) 3 Network for Computational Nanotechnology nanoHUB.org Adaptive VIOLINs online simulations and more Duke U. Internet Virtual clusters (VIOLINs) Physical cluster U. Florida nanoHUB infrastructure@Purdue 4 Network for Computational Nanotechnology nanoHUB.org Autonomic VIOLIN Adaptation online simulations and more • Adaptation triggers: Dynamic availability of infrastructural resources Dynamic resource needs of applications running inside • Adaptation actions: Resource re-allocation Scale adjustment (adding/deleting virtual machines) Re-location (migrating virtual machines) • Adaptation goals: Improving application performance Increasing infrastructural resource utilization Maintaining user/application transparency Minimizing infrastructure administrator attention 5 Network for Computational Nanotechnology nanoHUB.org Research Challenges online simulations and more • Autonomic live adaptation mechanisms VM Resource monitoring and scaling Application profiling and non-intrusive sensing of application needs Live VIOLIN re-location across domains • Adaptation policies VIOLIN adaptation model Infrastructure resource availability and topology Application resource needs Application configuration and topology Optimal VIOLIN adaptation decision-making Goals (cost vs. gains)? When to adapt? How and how much to adapt? Where to migrate? 6 Network for Computational Nanotechnology nanoHUB.org Overall Architecture online simulations and more VMs VMs VIOLIN Switch Dom0 VIOLIN Switch VIOLIN Switch Monitoring Daemon VIOLIN Switch Monitoring Daemon VMM Dom0 VMM VMs VMs Physical Network VIOLIN Switch VIOLIN Switch VIOLIN Switch CPU Update Dom0 Monitoring Daemon Monitoring Daemon Scale Migrate Up VMM Adaptation Manager 7 VMM Network for Computational Nanotechnology Dom0 nanoHUB.org VIOLIN Adaptation Policies online simulations and more • Maintain desirable resource utilization level • Reclaim resource if under-utilized over a period • Add resource if over-utilized over a period Scale up local resource share Migrate to other host(s) Balance host workload Intra-domain migration first Minimize migration • Re-adjust resource according to application needs 8 Network for Computational Nanotechnology nanoHUB.org Implementation and Deployment online simulations and more • Extension to non-adaptive VIOLIN Based on Xen 3.0 (w/ VM Live migration capability) Enabling live VIOLIN migration across domains IP addresses of VMs Root file systems of VMs Leveraging Xen libraries for VM resource monitoring (xenstat, xentop) Extending VIOLIN switch for inter-VM bandwidth monitoring • Deployment in nanoHUB On-line, on-demand simulation service for nanotechnology community Web interface for regular users “My workspace” interface for advanced users Local infrastructure: two clusters in two subnets 9 Network for Computational Nanotechnology nanoHUB.org nanoHUB Deployment Overview online simulations and more Local Virtual Machines Migratable VIOLIN Virtual Cluster Isolated from Local infrastructure Delegated trust Virtual Infrastructure over WAN 10 Network for Computational Nanotechnology nanoHUB.org VIOLIN in nanoHUB online simulations and more In the backround: Simulation job 11 VIOLIN Network for Computational Nanotechnology nanoHUB.org VIOLIN in nanoHUB online simulations and more Autonomic property: Users focus on simulation semantics and results, unaware of VIOLIN creation, setup, and adaptation. 12 Network for Computational Nanotechnology nanoHUB.org Impact of Migration on App. Execution online simulations and more End-to-end execution time of NEMO3D w/ and w/o live VIOLIN migration 13 Network for Computational Nanotechnology nanoHUB.org VIOLIN Adaptation Scenario online simulations and more 2. After 1. Initially VIOLIN VIOLIN 2 is 1,finished, 2, 3 are computing, VIOLINadaptation before 2 is about to be finished. Without Adaptation Domain 1 VIOLIN 1 VIOLIN 3 VIOLIN 2 VIOLIN 4 VIOLIN 5 With Adaptation Domain 2 Domain 1 14 Domain 2 Network for Computational Nanotechnology nanoHUB.org VIOLIN Adaptation Scenario online simulations and more 3. After adaptation 2. VIOLIN 2 is finished, before adaptation Without Adaptation Domain 1 VIOLIN 1 VIOLIN 3 VIOLIN 2 VIOLIN 4 VIOLIN 5 With Adaptation Domain 2 Domain 1 15 Domain 2 Network for Computational Nanotechnology nanoHUB.org VIOLIN Adaptation Scenario online simulations and more 4. After adaptation 3. VIOLIN 4, 5 are created Without Adaptation Domain 1 VIOLIN 1 VIOLIN 3 VIOLIN 2 VIOLIN 4 VIOLIN 5 With Adaptation Domain 2 Domain 1 16 Domain 2 Network for Computational Nanotechnology nanoHUB.org VIOLIN Adaptation Scenario online simulations and more 4. After VIOLIN 1, 5. 4, 3 5 are finished created Without Adaptation Domain 1 VIOLIN 1 VIOLIN 3 VIOLIN 2 VIOLIN 4 VIOLIN 5 With Adaptation Domain 2 Domain 1 17 Domain 2 Network for Computational Nanotechnology nanoHUB.org VIOLIN Adaptation Scenario online simulations and more 6. 5. ALL AfterVIOLINs VIOLIN 1, are 3 finished are finished Without Adaptation Domain 1 VIOLIN 1 VIOLIN 3 VIOLIN 2 VIOLIN 4 VIOLIN 5 With Adaptation Domain 2 Domain 1 18 Domain 2 Network for Computational Nanotechnology nanoHUB.org Limitations and Future Work online simulations and more • Simple, heuristic adaptation policy Application of machine learning and data mining techniques • Centralized adaptation manager Hierarchical or peer-to-peer adaptation managers • Imprecise application resource demand inference Multi-dimensional, fine-grain resource demand profiling • Campus-wide infrastructure Evaluation and deployment in wide-area infrastructure 19 Network for Computational Nanotechnology nanoHUB.org Related Work online simulations and more • • • • • VNET (Northwestern U.) Cluster-on-Demand (COD) (Duke U.) Virtual Workspaces on Grid (Argonne National Lab) SoftUDC (HP Labs) WOW and IPOP (U. Florida) 20 Network for Computational Nanotechnology nanoHUB.org Conclusions online simulations and more • Autonomically adaptive virtual infrastructures (VIOLINs) A new opportunity brought by virtualization technologies Decoupled from underlying shared infrastructure Intelligent, first-class entities with user-transparent resource provisioning • Key benefits Application performance improvement Infrastructure resource utilization Management convenience (at both virtual and physical levels) “The Cray motto is: adapt the system to the application - not the application to the system.” - Steve Scott, CTO, Cray Inc. on “adaptive supercomputing”, March 2006 21 Network for Computational Nanotechnology nanoHUB.org online simulations and more Thank you. For more information: Email: [email protected] URL: http://www.cs.purdue.edu/~dxu Google: “Purdue VIOLIN FRIENDS” 22 Network for Computational Nanotechnology