Download PPT

Stress Testing of
Research Systems
Ron Hood, Manager Voice/Data Operations
[email protected] / 206.667.5063
NYC BRIITE – May 4, 2006
Recent Painful Experiences
&
How I Hope to Avoid
Repeating Them in the
Future
Ron Hood, Manager Voice/Data Operations
[email protected] / 206.667.5063
NYC BRIITE – May 4, 2006
Hutch Background
•
•
•
•
Seattle, Washington
~2,500 Faculty & Staff
12 buildings on a 13 acre campus
Stand-alone institution (not part of a
university or hospital)
• Research only – no clinical care
NYC BRIITE – May 4, 2006
Hutch IT Background
•
•
•
•
Decentralized – 30 different IT groups.
“Center IT” is the largest @ ~60 FTE.
Housed in Administration division.
Desktop, File/Print, and Application support
for Admin & Clinical Research divisions.
• Project Management support for Center IT.
• Network, Information Security, and email for
entire center.
NYC BRIITE – May 4, 2006
Ron’s Background
• Has-been network geek.
• Newbie @ information security.
• Minimal knowledge of servers, desktops,
software development, project
management, and scientific applications.
• Responsible for voice, network, information
security, and physical infrastructure.
NYC BRIITE – May 4, 2006
Server Rooms: HVAC
NYC BRIITE – May 4, 2006
NYC BRIITE – May 4, 2006
Servers
NYC BRIITE – May 4, 2006
N
N
Servers
NYC BRIITE – May 4, 2006
UPS
UPS
N
N
Servers
NYC BRIITE – May 4, 2006
UPS
UPS
N
C
R
A
C
NYC BRIITE – May 4, 2006
N
Servers
C
R
A
C
UPS
UPS
N
C
R
A
C
NYC BRIITE – May 4, 2006
N
Servers
C
R
A
C
UPS
UPS
N
C
R
A
C
NYC BRIITE – May 4, 2006
N
Servers: 50 Cabinets
C
R
A
C
125 kW
125 kW
N
C
R
A
C
NYC BRIITE – May 4, 2006
N
Servers: 50 Cabinets
C
R
A
C
125 kW
125 kW
N
C
R
A
C
NYC BRIITE – May 4, 2006
N
Servers: 50 Cabinets
Power: 100 kW => 2 kW/Cabinet
C
R
A
C
125 kW
125 kW
N
2
0
T
NYC BRIITE – May 4, 2006
N
Servers: 50 Cabinets
Power: 100 kW => 2 kW/Cabinet
2
0
T
125 kW
125 kW
N
2
0
T
N
Servers: 50 Cabinets
Power: 100 kW => 2 kW/Cabinet
Cooling: 56 kW => ~ 1 kW/Cabinet
NYC BRIITE – May 4, 2006
2
0
T
Take-Away Lessons:
• Do the math.
NYC BRIITE – May 4, 2006
Server Rooms: Power
NYC BRIITE – May 4, 2006
NYC BRIITE – May 4, 2006
NYC BRIITE – May 4, 2006
Take-Away Lessons:
• Do the math.
• Understand the architecture.
NYC BRIITE – May 4, 2006
Storage
NYC BRIITE – May 4, 2006
Clinical
SAN in a Can
Clinical Tray
NYC BRIITE – May 4, 2006
Admin
Clinical
SAN in a Can
Clinical Tray
NYC BRIITE – May 4, 2006
Admin Tray
Admin
Clinical
SAN in a Can
Clinical Tray
NYC BRIITE – May 4, 2006
Admin Tray
Admin
Clinical
SAN in a Can
NVRAM
Clinical Tray
NYC BRIITE – May 4, 2006
Admin Tray
Take-Away Lessons:
• Do the math.
• Understand the architecture.
NYC BRIITE – May 4, 2006
Take-Away Lessons:
• Do the math.
• Understand the architecture.
NYC BRIITE – May 4, 2006
Networks
NYC BRIITE – May 4, 2006
NYC BRIITE – May 4, 2006
Take-Away Lessons:
• Do the math.
• Understand the architecture.
• Test regularly.
NYC BRIITE – May 4, 2006
NYC BRIITE – May 4, 2006
Take-Away Lessons:
•
•
•
•
Do the math.
Understand the architecture.
Test regularly.
Don’t touch it.
NYC BRIITE – May 4, 2006
Where do we go from here?
NYC BRIITE – May 4, 2006
Where do we go from here?
• We’ve learned some of the right
things to do.
NYC BRIITE – May 4, 2006
Where do we go from here?
• We’ve learned some of the right
things to do.
• It remains to be seen if we will have
the discipline to do them.
NYC BRIITE – May 4, 2006
Questions?
NYC BRIITE – May 4, 2006