Download ppt - Friends - Purdue University

nanoHUB.org
online simulations and more
IEEE International Conference on Autonomic Computing (ICAC’06)
Autonomic Live Adaptation of Virtual Computational
Environments in a Multi-Domain Infrastructure
Paul Ruth, Junghwan Rhee, Dongyan Xu
Department of Computer Science and Center for Education and
Research in Information Assurance and Security (CERIAS)
Rick Kennell, Sebastien Goasguen
Rosen Center for Advanced Computing
Purdue University
West Lafayette, Indiana, USA
1
Network for Computational Nanotechnology
nanoHUB.org
Outline of Talk
online simulations and more
•
•
•
•
•
•
•
Motivations
Overall architecture
Design and implementation
Real-world deployment in nanoHUB
Related work
Conclusion
Demo
2
Network for Computational Nanotechnology
nanoHUB.org
Motivations
online simulations and more
• Formation of shared distributed cyberinfrastructure (CI)
 Spanning multiple domains
 Serving users/user communities with diverse computation needs
 Exhibiting dynamic resource availability and workload
• Need for virtual distributed environments (VIOLINs), each with
 Customizability and legacy application compatibility
 Administrative privileges
 Isolation, security, and accountability
 Autonomic adaptation capability
- A unique opportunity brought by virtualization (VMs and VNs)
3
Network for Computational Nanotechnology
nanoHUB.org
Adaptive VIOLINs
online simulations and more
Duke U.
Internet
Virtual clusters
(VIOLINs)
Physical cluster
U. Florida
nanoHUB infrastructure@Purdue
4
Network for Computational Nanotechnology
nanoHUB.org
Autonomic VIOLIN Adaptation
online simulations and more
• Adaptation triggers:
 Dynamic availability of infrastructural resources
 Dynamic resource needs of applications running inside
• Adaptation actions:
 Resource re-allocation
 Scale adjustment (adding/deleting virtual machines)
 Re-location (migrating virtual machines)
• Adaptation goals:




Improving application performance
Increasing infrastructural resource utilization
Maintaining user/application transparency
Minimizing infrastructure administrator attention
5
Network for Computational Nanotechnology
nanoHUB.org
Research Challenges
online simulations and more
• Autonomic live adaptation mechanisms
 VM Resource monitoring and scaling
 Application profiling and non-intrusive sensing of application needs
 Live VIOLIN re-location across domains
• Adaptation policies
 VIOLIN adaptation model
 Infrastructure resource availability and topology
 Application resource needs
 Application configuration and topology
 Optimal VIOLIN adaptation decision-making
 Goals (cost vs. gains)?
 When to adapt?
 How and how much to adapt?
 Where to migrate?
6
Network for Computational Nanotechnology
nanoHUB.org
Overall Architecture
online simulations and more
VMs
VMs
VIOLIN Switch
Dom0
VIOLIN Switch
VIOLIN Switch
Monitoring Daemon
VIOLIN Switch
Monitoring Daemon
VMM
Dom0
VMM
VMs
VMs
Physical
Network
VIOLIN Switch
VIOLIN Switch
VIOLIN Switch
CPU Update
Dom0
Monitoring Daemon
Monitoring Daemon
Scale
Migrate
Up
VMM
Adaptation
Manager
7
VMM
Network for Computational Nanotechnology
Dom0
nanoHUB.org
VIOLIN Adaptation Policies
online simulations and more
• Maintain desirable resource utilization level
• Reclaim resource if under-utilized over a period
• Add resource if over-utilized over a period
 Scale up local resource share
 Migrate to other host(s)
 Balance host workload
 Intra-domain migration first
 Minimize migration
• Re-adjust resource according to application needs
8
Network for Computational Nanotechnology
nanoHUB.org
Implementation and Deployment
online simulations and more
• Extension to non-adaptive VIOLIN
 Based on Xen 3.0 (w/ VM Live migration capability)
 Enabling live VIOLIN migration across domains
 IP addresses of VMs
 Root file systems of VMs
 Leveraging Xen libraries for VM resource monitoring (xenstat, xentop)
 Extending VIOLIN switch for inter-VM bandwidth monitoring
• Deployment in nanoHUB
 On-line, on-demand simulation service for nanotechnology community
 Web interface for regular users
 “My workspace” interface for advanced users
 Local infrastructure: two clusters in two subnets
9
Network for Computational Nanotechnology
nanoHUB.org
nanoHUB Deployment Overview
online simulations and more
Local Virtual Machines
Migratable
VIOLIN Virtual
Cluster
Isolated from Local
infrastructure
Delegated trust
Virtual
Infrastructure
over WAN
10
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN in nanoHUB
online simulations and more
In the backround:
Simulation
job
11
VIOLIN
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN in nanoHUB
online simulations and more
Autonomic property:
Users focus on simulation
semantics and results,
unaware of VIOLIN creation,
setup, and adaptation.
12
Network for Computational Nanotechnology
nanoHUB.org
Impact of Migration on App. Execution
online simulations and more
End-to-end execution time of NEMO3D w/ and w/o live VIOLIN migration
13
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN Adaptation Scenario
online simulations and more
2. After
1.
Initially
VIOLIN
VIOLIN
2 is
1,finished,
2, 3 are computing,
VIOLINadaptation
before
2 is about to be finished.
Without Adaptation
Domain 1
VIOLIN 1
VIOLIN 3
VIOLIN 2
VIOLIN 4
VIOLIN 5
With Adaptation
Domain 2
Domain 1
14
Domain 2
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN Adaptation Scenario
online simulations and more
3. After adaptation
2.
VIOLIN 2 is finished,
before adaptation
Without Adaptation
Domain 1
VIOLIN 1
VIOLIN 3
VIOLIN 2
VIOLIN 4
VIOLIN 5
With Adaptation
Domain 2
Domain 1
15
Domain 2
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN Adaptation Scenario
online simulations and more
4. After adaptation
3.
VIOLIN 4, 5 are created
Without Adaptation
Domain 1
VIOLIN 1
VIOLIN 3
VIOLIN 2
VIOLIN 4
VIOLIN 5
With Adaptation
Domain 2
Domain 1
16
Domain 2
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN Adaptation Scenario
online simulations and more
4. After VIOLIN 1,
5.
4, 3
5 are finished
created
Without Adaptation
Domain 1
VIOLIN 1
VIOLIN 3
VIOLIN 2
VIOLIN 4
VIOLIN 5
With Adaptation
Domain 2
Domain 1
17
Domain 2
Network for Computational Nanotechnology
nanoHUB.org
VIOLIN Adaptation Scenario
online simulations and more
6.
5. ALL
AfterVIOLINs
VIOLIN 1,
are
3 finished
are finished
Without Adaptation
Domain 1
VIOLIN 1
VIOLIN 3
VIOLIN 2
VIOLIN 4
VIOLIN 5
With Adaptation
Domain 2
Domain 1
18
Domain 2
Network for Computational Nanotechnology
nanoHUB.org
Limitations and Future Work
online simulations and more
• Simple, heuristic adaptation policy
 Application of machine learning and data mining techniques
• Centralized adaptation manager
 Hierarchical or peer-to-peer adaptation managers
• Imprecise application resource demand inference
 Multi-dimensional, fine-grain resource demand profiling
• Campus-wide infrastructure
 Evaluation and deployment in wide-area infrastructure
19
Network for Computational Nanotechnology
nanoHUB.org
Related Work
online simulations and more
•
•
•
•
•
VNET (Northwestern U.)
Cluster-on-Demand (COD) (Duke U.)
Virtual Workspaces on Grid (Argonne National Lab)
SoftUDC (HP Labs)
WOW and IPOP (U. Florida)
20
Network for Computational Nanotechnology
nanoHUB.org
Conclusions
online simulations and more
• Autonomically adaptive virtual infrastructures (VIOLINs)
 A new opportunity brought by virtualization technologies
 Decoupled from underlying shared infrastructure
 Intelligent, first-class entities with user-transparent resource provisioning
• Key benefits
 Application performance improvement
 Infrastructure resource utilization
 Management convenience (at both virtual and physical levels)
“The Cray motto is: adapt the system to the application - not the
application to the system.”
- Steve Scott, CTO, Cray Inc. on “adaptive supercomputing”, March 2006
21
Network for Computational Nanotechnology
nanoHUB.org
online simulations and more
Thank you.
For more information:
Email: [email protected]
URL:
http://www.cs.purdue.edu/~dxu
Google: “Purdue VIOLIN FRIENDS”
22
Network for Computational Nanotechnology