Download Nonstop computing

Nonstop
computing
Non-stop
computing
學 生:R90725004 黃錫煙
:R90725007 黃春照
指導教授:
莊 裕 澤 教授
Nonstop computing
Agenda
 E*Trade Case Study
Oracle9i Real Application Cluster
 Sun Cluster [TM] 3.0 architecture
 Sun Cluster [TM] 3.0 management
 Conclusion
 Reference
Nonstop computing
E*Trade Team
●
Strong management, marketing and technology
teams from world leaders such as Schwab, Fedex,
Chase, Visa.
●
VP of Engineering: 10 years VISA system
development and operation experience
●
10 Core development engineers are also from VISA,
have the know-how of building high volume
transaction systems
●
Web creative team: experienced in developing
attractive and user-friendly web interface for on-line
trading
Nonstop computing
E*Trade Challenges
●
Customer retention
- service personalization
- information customization
- dynamic content generation
●
Peak load handling
- There is no way to control the internet user base
- There is no way to control concurrent users
- System must be able to scale up quickly to meet
growing customer base
●
No trade drop
- Highly reliable system
- Disaster recovery capability
- Transactional integrity
Nonstop computing
E*Trade Challenges
●
Must bring out new products and services
quickly to meet market demand and stay
ahead of the competition
- a flexible architecture
- a rapid development environment
●
Integration with partner offerings
- 3rd party integration capability
●
24x7 availability
- No service downtime even when new services are
launched
- No service downtime even when new hardware are
added
- Innovative off-hour services
Nonstop computing
E*Trade - Today
100% Replace the existing CGI based Stock Trading
services with NAS and Java based Solutions
● Over 10 groups, about 70 people working on NAS
based projects.
- US Stock Trading
●
-
-
Risk management
Research tools
Portfolio management tools
Advertisement
Customer Care
Marketing
Professional Edge
Smart Alert
Content Alliances
…
Nonstop computing
E*Trade - Today
●
Scalable and High Performance:
-
●
Open:
-
●
redundant on-line systems plus two identical data centers
Reliable:
-
●
works with but not lock in with any hardware platforms,
DBs or clients
Highly available:
-
●
multi-tier architecture
simulated 100,000 simultaneous concurrent users
even if the west coast data center is down, the incomplete
transactions will be rolled over to the east coast data
center over a WAN to continue the execution
Flexible:
-
layered, modularized application design
new service launch every two weeks
easy partner offering integration
Nonstop computing
Software Architecture
Clients
Comm
GW
Stateful
Apps
Stateless
Apps
Database
Browser
WebTV
PDA
I18n connection
Quote
News
HTTP
IVR
IIOP
Sell
OFX
Thin Clients
Digital Certificates
Buy
NAS: State/session Fail-over Load balancing
Order
mngt
Quote
mngt
Content
mngt
Acct
mngt
Tuxedo: TP Monitor
Stock
Exchange
SDK
Data Feeds
Nonstop computing
System Architecture
Nonstop computing
E*Trade Success
●
Business Benefits
- Created significant new business
opportunity in less time
- Extends E*Trade’s position as
“premier” electronic brokerage
firm
- Cost effective, reliable hardware
infrastructure
Debra Chrapaty
Chief Technology Officer
Nonstop computing
Oracle9i
Cluster Strategy
●
Faster Database Software
-
●
Run Database on Multiple Computers
Faster Database Computer
- More Processors – 2x64 or 32x4
- Faster Processors – 4 @ 700 MHz
●
No Single Point of Failure
Nonstop computing
Oracle9i
Cluster Strategy
●
Database Software on Multiple
Computers
- Very Hard to do for Real Applications
- Everyone did it for Benchmarks – TPC-C
- Only IBM Mainframes did it for
Real Applications
Nonstop computing
Oracle9i
Cluster Types
• Shared Disk
•
Shared Nothing
Runs Real Applications
Only Runs Benchmarks
– IBM DB2 for Mainframes – IBM DB2 UNIX & Windows
– Oracle9iDB
– Microsoft SQLServer
Data
A-Z
Data
A-E
Data
F-K
Data
L-S
Data
T-Z
Nonstop computing
Oracle9i
Shared Nothing Clusters
IBM UNIX & Windows, MS
SQLServer
●
Runs No Existing Applications
- You Must Write Custom Applications
●
Gets Less Reliable As You Add Computers
- Multiple Single Points of Failure
●
Very Hard to Manage
-
You Must Re-Partition Data To Add Computers
Nonstop computing
Oracle9i Real Application Clusters
●
Unlimited Reliability
- More Computers = More Reliable
Application
Oracle Real Application Cluster
Nonstop computing
Oracle9i
Oracle9i Unique New Features
• Availability DataGuard On-Line
Reorganization Flash-Back Query
• Hosting
– Enhanced Virtual Private
Database
• Security
– Label Security
– Fine Grained Auditing
• Management
– Feedback-Directed Memory
–
Management
Resumable Statements
• Business Intelligence
– Integrated OLAP
– Advanced Analytics
– ETL
– Bitmap-Join Indexes
• Content Management
– Files
– UltraSearch
• Web Services
– Dynamic Services
– XML Datatype
Nonstop computing
Oracle9i
Oracle Data Guard
Oracle9i
Standby System
Reporting
from Standby
Broker
Production
Database
Redo
Logs
Log Ship
Oracle9i
Backup
from Standby
Standby
Database
Log Receive
Standby
Logs
Broker Management Interface
Log Apply
Optional
Delay
Broker
Primary System
Nonstop computing
Oracle9i
Oracle9i Database Online
Maintenance
Reducing Planned Downtime
Operation/Time to Complete
Oracle9i
IBM
Microsoft
Re-organize 100GB Table
with indexes
online
16 hours
18 hours
Undo accidental delete from 8
hours ago
1 min.
14 hours
14 hours
Add additional node to cluster
and re-distribute data
online
26 hours
Can’t
Nonstop computing
Traditional Business
Intelligence
Oracle9i
Extract
Analyze
Stage
Loa
d
Stor
e
Transform
Use
Mine
Nonstop computing
Oracle9i - Better business decisions faster
Extract
Stage
Load
Transform
Store
Analyze
Mine
Use
Nonstop computing
Oracle9i Database - ensures your
business information is always
available
System
Failures
Unplanned
Downtime
Data Failures
& Disasters
Human
Errors
System
Maintenance
Planned
Downtime
Database
Maintenance
Real Application Clusters
Continuous Availability for all Applications
Data Guard
Guaranteed Zero Data Loss
Flashback Query
Enable Users to Correct their Mistakes
Dynamic Reconfiguration
Capacity on Demand without Interruption
Online Redefinition
Adapt to Change Online
Nonstop computing
Oracle9i
Improved Data Warehousing
Operation in Hours:Mins
Oracle8i
Oracle9i
Gain
Load (Load, Merge, Multi Insert)
6:00
2:11
x 2.75
Transform (Multi Stage, Parallel)
8:00
1:04
x 7.45
Store (Summarize, Refresh)
8:00
0:17
x 28
Analyze (Group, Rollup, Cube)
12:00
2:24
x5
Mine (xxxx ?)
6:00
1:12
x5
TOTAL
40:00
7:08
x5-6
Do overnight what used to take a weekend.
Nonstop computing
Oracle9i Database
Option and Edition Availability
Option/Availability
Personal
Standard
Enterprise
Oracle Real Application Clusters
No
No
Yes
Oracle Advanced Security
Yes
No
Yes
Oracle Label Security
Yes
No
Yes
Oracle Partitioning
Yes
No
Yes
Oracle OLAP
Yes
No
Yes
Oracle Data Mining
Yes
No
Yes
Oracle Spatial
Yes
No
Yes
Oracle Programmer
Yes
Yes
Yes
Nonstop computing
Choosing A Deployment Platform
A Single SMP
• Scales to
multiple CPUs
• Doesn’t scale
beyond one
node
• Multiple single
points of failure
Users
Nonstop computing
Choosing A Deployment Platform
Failover
Clustering
• Fault tolerant
systems; highly
available
• Doesn’t scale
beyond one
node
Users
Oracle9i
Nonstop computing
Oracle Parallel Server
Grow your data
Grow your users
Users
Grow processing
power
Oracle9i
Nonstop computing
Real Application Clusters
Grow your data
Users
Grow your users
Grow processing
power
Cache Fusion:
Performance of a
shared cache
Oracle9i
Nonstop computing
Announcing Oracle9iDB
Real Application Clusters
●
●
●
●
Runs SAP
Runs PeopleSoft
Runs Siebel
Runs All Your Applications
Oracle9i
Nonstop computing
Sun Cluster [TM] 3.0 architecture
Sun Cluster [TM] 3.0 management
Nonstop computing
Types of Clusters
Plexes - Hardware Components
Network
Servers/Domain
Domain 4
Domain 3
Domain 2
Domain 1
Storage
Public
Network
Interconnects
A Domain is an instance of Solaris software running on an isolated set
of hardware : CPUs, memory, and I/O.
Nonstop computing
Types of clusters
Plexes – Software components
Cluster Software: Userland software
Integrated with Solaris
Applications
Domain 4
Agents
Domain 3
Domain
3 2
Domain
Domain 1
Network
Cluster
Software
Solaris
- Sun Cluster2.2, VCS 1.x
- Sun Cluster 3.0
Nonstop computing
Types of Clusters
Plexes
0 – What is Tight Coupling?
5
VCS 1.x Sun Cluster 2.2
1
Sun Cluster 3.0
0
Tight Coupling scale 0-10
• Interconnects – low latency, high-bandwidth links
• Type of interconnects technology
➔
Fast Ethernet, Gigabit Ethernet, SCI, Future
• Number of interconnects between nodes
➔
➔
➔
Sun Cluster 3.0 : (min 2, max 6)
Sun Cluster 2.2 : (min and max 2)
VCS 1.x
: (min 1 and max 2)
• Solaris OE and cluster software interface
• Integrated
- Sun Cluster 3.0 (enables SunPlex environments)
• Userland
- Sun Cluster 2.2 and VCS 1.x
Nonstop computing
SunPlex
Enabled by Sun Cluster 3.0
Application Service
Sun Cluster 3.0
Solaris OE Cluster Extensions
Nonstop computing
SunPlex
Conceptual breakthrough!
Application
Service
Services
Global Network
Service
Global File
Service
Sun Cluster 3.0
Solaris OE Cluster Extensions
Three core application service components are decoupled.
Nonstop computing
SunPlex
Physical View of Core Application Service Components
Web
Global File Service
DB
Web
Web
Domain 6
Web
Domain 5
Domain 4
Web
Domain 3
DB
Domain 2
Domain 1
Global Network Service
Nonstop computing
SunPlex
Service Level Management Platform to Deliver Applications
Scalable
Services
Failover
Services
Nonstop computing
SunPlex
Failover Service Example – Databases
Global Network Service
Global File Service
DB
Domain 6
Domain 5
Domain 4
Domain 3
DB
Domain 2
Domain 1
DB
Failover
Service
Nonstop computing
SunPlex
Cluster Aware Service Example – Oracle 9i Real
Application Cluster
Global Network Service
RAC
3
Domain 4
RAC
2
Domain 3
RAC
1
Domain 2
Domain 1
RAC
RAC
4
Oracle Data Files on shared Disks
Oracle 9i
Real
Application
Cluster
Instance
Nonstop computing
SunPlex
Cluster Aware Service Example – Oracle 9i Real
Application Cluster
Global Network Service
RA
C3
Domain 4
RAC
2
Domain 3
RAC
1
Domain 2
Domain 1
RAC
RAC
4
Oracle Data Files on shared Disks
Oracle 9i
Real
Application
Cluster
Instance
Nonstop computing
SunPlex
Scalable Service Example – Web Server
Global Network Service
Global File Service
DB
Web
Domain 6
Web
Domain 5
Web
Domain 4
Web
Domain 3
DB
Domain 2
Domain 1
Web
Web
Scalable
Service
Nonstop computing
SunPlex
Scalable Service Example – Web Server
Global Network Service
Web
Global File Service
DB
Web
Domain 6
Web
Domain 5
Web
Domain 4
Web
Domain 3
DB
Domain 2
Domain 1
Web
Web
Scalable
Service
Nonstop computing
Application Services & Resources
Resource Groups view in a cluster – Failover and Scalable
Node 1
Node 2
Failover
Resource
Group
Interconnect
Scalable
Resource
Group
Scalable
Resource
Group
Shared
Storage
Nonstop computing
Sun Cluster 3.0 Main Components
Solaris 8 OE
HA Framework
Global Components
Userland Components
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components : Devices
Tape Drive
Domain 4
Domain 3
Domain 2
Domain 1
Network
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components – File Service
Global File Service
Domain 4
Domain 3
Domain 2
Domain 1
Network
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components – File Service
Global File Service
Domain 4
Domain 3
Domain 2
Domain 1
Network
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components – File Service Offers
●
●
●
●
A continous availability of data
Sun Cluster 3.0 maintains the access to the data
No changes in file systems required
No changes in applications accessing file system
data required
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components – Network Service
Global File Service
Domain 4
Domain 3
Domain 2
Domain 1
Global LAN/WAN
Network Service
Nonstop computing
Sun Cluster 3.0 Main Components
Global Components – Network Service
Global File Service
Domain 4
Domain 3
Domain 2
Domain 1
Global LAN/WAN
Network Service
Nonstop computing
Sun Cluster 3.0 Architecture
Architecture
Agents
API
Resource Group Mgr
User
Kernel
Public Network
Monitor
Global Network
Service
Cluster Membership
Monitor
Cluster Configuration
Repository
Global File Service
Volume Mgt
Storag
TCP/IP
N/W
stack
Cluster
Transport
Global Device
Access
HA
Framework
Other
Nodes
Nonstop computing
Sun Cluster 3.0 Architecture
HA Framework Components
• Cluster Transport
• Cluster Configuration Repository
• Cluster Membership Monitor
Cluster Membership
Monitor
Cluster Configuration
Repository
Cluster
Transport
HA Framework
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components : Solaris Software File Service
Application
●
File System
Volume Manager
Storage
Solaris Software File
Service: Implemented at the
vnode layer
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global File Service (GFS)
Application
vnode layer
Global File Service
vnode layer
File System
Volume Manager
Storage
Single Node View of GFS
Nonstop computing
Sun Cluster 3.0 architecture
Global Feature Components: Global File Service (GFS)
Application
Application
vnode layer
Global File Service
Global File Service
vnode layer
File System
Volume Manager
Cluster View of GFS
Storage
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global File Service (GFS)
Application
●
●
Global File Service
File System
●
●
●
Volume Manager
●
*NOT* a new file system
PxFS Mechanism built at vnode
interface of Solaris OE
Independent of file system type &
volume manager
Coherency protocol preserves
UNIX file access semantics
Failover/switchover transparent to
user & application
mount -g once and access
anywhere in cluster
Storage●
/etc/vfstab global mount option
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: How GFS Works?
Node 2
Node 1
GFS
GFS
Cluster Interconnect
Cluster
Transport
Transport
PxFS Secondary
PxFS Primary
File
System
Shared
Storage
Volume Manager
Nonstop computing
Sun Cluster 3.0 architecture
Global Feature Components: How GFS works?
Node 2
Node 1
GFS
GFS
Cluster Interconnect
Cluster
Transport
Transport
PxFS primary
File
System
Volume Manager
Shared
Storage
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global Network Service
(GNS) Shared IP Address
Network Client
for App 1
Network Client
for App 2
Shared IP Address
App 1
Data 1
App 2
Data 2
Sun Cluster 3.0 Environment
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global Network Service (GNS)
Shared IP Address
GNS
Shared IP Address
Resource: IP1
App 1
GNS
App 2
Node 1
Node 2
Data 1
Data 2
Shared IP Address -- relocatable cluster-wide resource
●Application -- independent of IP address
●Multiple applications -- use same IP address running on the
same node or cluster wide
●
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: How GIF Works?
Request/Reply
for
App 1 Service
NIC 1
(GIF)
GNS
App 1
App 1
Client
Shared IP
Resource
Interconnect
Node 1
App 2
Client
GNS
NIC 2
(non-GIF)
App 2
Node 2
Data 1
Data 2
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: How GIF Works?
App 1
Client
NIC 1
(GIF)
App 2
Client
Request
for
App 2 Service
GNS
App 1
Shared IP
resource
Interconnect
Node 1
GNS
NIC 2
(non-GIF)
App 2
Node 2
Data 1
Data 2
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: How GIF Works?
App 1
Client
NIC 1
(GIF)
GNS
App 1
Shared IP
resource
Interconnect
Node 1
App 2
Client
GNS
Reply
For
App 2
Service
NIC 2
(non-GIF)
App 2
Node 2
Data 1
Data 2
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global Network Service
Scalable Services
Data on Global
File Service
Domain 6
App
3
Domain 5
App
2
Domain 3
Ap
App
p1
1
Domain 2
Domain 1
GIF
Network Card
NIC 2 & 3
(Not GIFs)
Domain 4
Application
Clients on the
Network
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global Network Service
Scalable Services
Data on Global
File Service
Domain 6
App
3
Domain 5
App
2
Domain 3
Ap
App
p1
1
Domain 2
Domain 1
GIF
Network Card
NIC 2 & 3
(Not GIFs)
Domain 4
Application
Clients on the
Network
Nonstop computing
Sun Cluster 3.0 Architecture
Global Feature Components: Global Network Service
Scalable Services
Data on Global
File Service
Domain 6
App
3
Domain 5
App
2
Domain 3
Ap
App
p1
1
Domain 2
Domain 1
GIF
Network Card
NIC 2 & 3
(Not GIFs)
Domain 4
Application
Clients on the
Network
Nonstop computing
Sun Cluster 3.0 Architecture
Userland Components:
• RGM – Resource Group Manager
• Public Network Monitoring NAFO
• Agents/Applications
• API
Agents/Applications
API
Resource Group Mgr
User
Kernel
Public Network
Monitor
Nonstop computing
Sun Cluster 3.0 Management
Monitoring of Sun Cluster 3.0
• Single monitoring view for Sun Cluster
• Monitors cluster components
• Nodes, interconnects and device groups
• RGM resource monitoring and creation
• Quorum devices
• Cluster Status
Nonstop computing
Conclusion
• Innovative technology delivered in a powerful and
comprehensive cluster solution with ease of
operations and deployment
Nonstop computing
References
For more information ....
• http://www.compaq.com/buynow/o
nline.html
• http://www.ibm.com
• http://docs.sun.com
• Architecture and API Whitepapers
http://www.sun.com/clustersBlueP
rint : Desiging Enterprise
Solutions with Sun [tm] Cluster 3.0
ISBN # 0-13-008458-1
http://www.sun.com/blueprints