Download IPv6 - Internet2

IPv6: DoD Pilot Implementation
on DREN
Joint Techs Workshop
July 2004
Columbus, OH
Ron Broersma
DREN Chief Engineer
High Performance Computing Modernization Program
[email protected]
July 20, 2004
IPv6: DoD Pilot - DREN
1
Context for this briefing
• Historical
– June 2003 – DoD CIO issues IPv6 transition memorandum
• Target completion: 2008
– July 2003 – DREN chosen as the DoD IPv6 “pilot”
implementation
• Plans to implement in 2004
• Within DoD…
– Each of the services (Army, Navy, Air Force) developing their
own transition plans for the “operational networks”.
• Most will not begin implementation for a year or more
• Most will not be complete until after 2008
– DREN is DoD’s “research network”, and is transitioning now.
• Chartered to support the DoD HPC community, and other R&D
organizations.
July 20, 2004
IPv6: DoD Pilot - DREN
2
DREN Today
• 10 “core nodes” on OC-48 backbone (CONUS), with
extensions to Hawaii and Alaska.
– Now updating to OC-192 (10 Gigabit)
• About 100 sites (“Service Delivery Points”),
connected at DS-3 to OC-48 rates.
• IPv4 unicast and multicast, IPv6 unicast, and ATM
services now.
• Dual IPv6 networks (“testbed”, and “production”)
• “jumbo-clean” (i.e. 9K MTU everywhere)
• Multiple security levels.
– Both unclassified and classified networks
July 20, 2004
IPv6: DoD Pilot - DREN
3
DREN Map
July 20, 2004
IPv6: DoD Pilot - DREN
4
DREN IPv6 History
•
1995-2000
•
Jan 2001 -
–
–
–
–
Ad-hoc tunnels, playing on 6bone.
Presentation at conferences
IPSEC (NRL)
Early implementations (NRL stack)
–
DRENv6 “testbed”
–
DREN sites encouraged to connect and participate in testing and experimentation.
Many tests conducted, many lessons learned.
•
•
•
•
•
•
Native IPv6 (no tunnels)
Logically separate from DREN IPv4 backbone
OC-3 interconnects (ATM PVC mesh)
8 core nodes (Cisco routers – dedicated to IPv6)
Sites connect via PVCs (native IPv6), or tunnels.
Peering with IPv6 enabled ISPs
•
“If you build it, they will come”
•
2002
•
Jul 2003
•
Oct 2003
–
New DREN2 backbone contract (MCI) includes IPv6
–
Selected as DoD IPv6 “pilot” (details below)
–
Added DRENv6 node at Ft Huachuca (TIC, JITC) for Moonv6 interconnect between
DoD and Abilene (UNH)
July 20, 2004
IPv6: DoD Pilot - DREN
5
DRENv6 “testbed”
Logical Topology
Cisco
AIX-v6
C&W
Global
Crossing
FIX-West
Hurricane
Electric
LAVAnet
TIC
NTTCom
Verio
Abilene
6TAP
Abilene
WPAFB
Dayton
ARL
JITC
HP
San Diego
WCISD
SD-NAP
SDSC
SSC San Diego
Aberdeen
Tunnel broker
AOL
Wash D.C.
HICv
6
NRL
Vicksburg
(Hawaii)
SSAPAC
SPRINT
Albuquerque
AFRL
Kirtland AFB
ATM PVC (OC-3)
tunnel
July 20, 2004
SSC Charleston
ERDC
Stennis
NAVO
IPv6: DoD Pilot - DREN
vBNS+
IXP
Core Router
ISP or
BGP Neighbor
“site”
6
Lessons from Testbed experience
(state of things 1 year ago)
• Our customer sites find little or no incentive to run
IPv6 (LAN administrator perspective).
– There is no capability or feature of the Internet that you
can't do today by not running IPv6.
– Turning it on brings additional complexity, and has a learning
curve.
– Users aren’t asking for IPv6.
– There is no immediate "win" to transitioning to the new
protocol. The payoff is long-term. External incentives will
be needed to encourage near term adoption and transition.
• “If you build it, they won’t necessarily come”
• Many commercial security components (like Intrusion
Detection Systems, Firewalls, Security Scanners, etc.)
don't yet support IPv6, so it is very difficult to deploy
the technology to our sensitive DoD networks in a
secure fashion.
July 20, 2004
IPv6: DoD Pilot - DREN
7
DREN as DoD IPv6 Pilot
• DREN is in a unique position to serve as a
DoD IPv6 pilot
– Experience running IPv6 WAN.
– R&D environment – familiar with technology
insertion, and being a pioneer.
– New contract includes IPv6 support in the WAN
(we just have to turn it on).
– Management support.
– Have the means to deal with the challenges.
July 20, 2004
IPv6: DoD Pilot - DREN
8
FY04 DREN IPv6 Initiative
•
•
DoD IPv6 Pilot network
Goals for 2004
1.
4.
5.
IPv6 enabled DREN infrastructure (all Service Delivery Points, the
Wide Area Network, the NOC).
Facilitate IPv6 deployment into infrastructure at HPC user sites
and DREN user sites.
IPv6 enabled HPCMPO, HPCMP funded assets and services,
HPCMP user community support applications, selected user
application candidates.
Performance and Security as good as existing IPv4 service.
Provide product feedback, lessons learned, published via web.
–
–
–
–
–
–
–
IP transport and infrastructure
Infrastructure services
Network Management
Security
Applications
Planning for the Future
HPC Community Involvement
2.
3.
•
Functional Areas in this project:
July 20, 2004
Ron Broersma, Navy
Phil Dykstra, WCI
Tom Kile, Army
Doug Butler, OSD
Ralph McEldowney, Air Force
Ron Broersma, Navy
John Baird, OSD
IPv6: DoD Pilot - DREN
9
Transition Strategy (Notional)
•
•
•
Start with core, and work out to the edge
Hybrid (Dual Stack) infrastructure
Minimize need for tunnels, translators, and other transition schemes
SA
A
S
A
S
A
S
Site
LAN
Application
Server
July 20, 2004
Site
LAN
Site
LAN
S
A
S
Site
LAN
WAN (DREN)
NOC
Internet
IPv6: DoD Pilot - DREN
10
Goal #1: IPv6 enabled DREN infrastructure (all Service
Delivery Points, the Wide Area Network, the NOC).
Complete
• All 100+ WAN routers (Juniper) upgraded to JunOS 6.1 to
support IPv6.
– Includes all Service Delivery Points (SDPs) and DREN Core Nodes
(DCNs).
• Connectivity to Internet (IPv6) via DREN Testbed.
• Backbone is now IPv6 enabled and ready to bring production
sites online.
– Sites already turned up: HPCMO, SSC San Diego, ARL, NRL, ERDC,
Indian Head, Quantico, Norfolk, Charleston, DREN NOC.
• Tunnel Brokers (Hexago) for each network.
– Testbed, DREN, S/DREN
• Network and Users conferences are IPv6 enabled.
• Cleanup: readdressed entire WAN to conform to new
addressing plan.
July 20, 2004
IPv6: DoD Pilot - DREN
11
Goal #2: Facilitate IPv6 deployment into infrastructure
at HPC user sites and DREN user sites.
Complete (at HPC sites)
• “Road show” to 13 sites (to date)
– ARL, ASC, ERDC, NAVO, AHPCRC, ARSC, MHPCC, SMDC,
NRL-DC, RTTC, HPCMPO, DREN NOC, HPC CERT.
• Briefing for Executives, Management, and technical
staff.
–
–
–
–
Get buy-in from all levels of management.
Incentivise sites to upgrade local infrastructure and systems.
Offer assistance, resources, training.
Establish transition team within each organization.
• ASC went “live” on 26 June. ARL in August. Others
to follow.
July 20, 2004
IPv6: DoD Pilot - DREN
12
HPC sites being IPv6 enabled
ARSC
AHPCRC
ARL
ASC
NRL-DC
SMDC
Legend:
WSMR
RTTC
SSCSD
ERDC
Legend:
“Allocated” DCs
NAVO
“Dedicated”
“Allocated” DCs
DCs
“MSRCs”
July 20, 2004
MHPCC
IPv6: DoD Pilot - DREN
13
New Challenge
• Before:
– Little incentive to transition to IPv6
• Now:
– No real resistance.
– Site visits are paying off.
• New Problem:
– Transition to IPv6 is just one of many new
priorities (security, new systems, etc).
– Efforts with near term return on investment (ROI)
get priority. IPv6 transition has far term ROI.
July 20, 2004
IPv6: DoD Pilot - DREN
14
Goal #3: IPv6 enabled HPCMPO, HPCMP funded assets
and services, HPCMP user community support
applications, selected user application candidates.
Continuing Effort
• HPC Program office
– done
• HPC assets/services
– first ones starting to go live now
• HPC support applications
– Kerberos – mostly complete
– IDS – done
– Web sites (InfoEnv, OKC) – Fall ‘04
• User applications (mostly 3rd party)
– Discovery process well along
– Actual transition depends on vendor/developer
– Recent breakthrough: FlexLM (Macrovision) committed to IPv6
support
July 20, 2004
IPv6: DoD Pilot - DREN
15
Goal #4: Performance and Security as good as existing
IPv4 service
Success
• Performance:
– IPv6 performance within 0.3% of IPv4 on various
stress tests.
• Security
– Through workarounds, we can achieve equivalent
security posture.
– Catching attacks, blocking viruses.
– DSAWG Review: “no issues”.
July 20, 2004
IPv6: DoD Pilot - DREN
16
Performance Results
• Phil Dykstra (on DREN2 “pilot” net):
– “Using iperf, SSC [San Diego, CA] to ARL
[Aberdeen, Maryland], MTU 9k, I get about 567
Mbps with IPv4, 565 Mbps with IPv6. So at first
glance, performance seems nearly identical (minus
the extra header overhead of course).”
– Done between 2 Linux machines on opposite
coasts connected to DREN OC-12 sites.
• 10Gb-E testing at HPC Center, sending a 4 Gb/s
stream from Linux with 10Gb-E NIC.
– 3939.8044 Mbps UDP single stream (IPv4)
– 3930.6234 Mbps UDP single stream (IPv6)
July 20, 2004
IPv6: DoD Pilot - DREN
17
DoD Security Model
• “Defense in Depth”
– Protections at multiple
levels
• Problem: How to
securely deploy IPv6
in DoD without these
components.
S
Scanners
LAN
Firewall
IDS
ACL
WAN
ACL
IDS
Internet
July 20, 2004
IPv6: DoD Pilot - DREN
18
Lack of Security Features
(Examples)
•
Router Access Control Lists (ACLs)
•
Vulnerability Assessment (Scanners)
•
Intrusion Detection Systems
•
IPSEC
•
Firewalls
– Juniper doesn’t support “tcp established”
– ISS doesn’t support IPv6 and has no published plans to do so.
– NESSUS doesn’t support IPv6 (yet)
– If we want IPv6 support, we have to add it ourselves.
– Juniper port mirroring doesn’t support IPv6
– Missing in most IPv6 implementations
– Juniper ASPIC doesn’t support IPv6 (until much later)
– Until recently, no production quality IPv6 support
– Netscreen (Juniper):
• no OSPFv3, only RIP
• IPv6 support only available in certain products
– High end products won’t have IPv6 support until next year.
It is crucial that IPv6 products have equivalent functionality to the IPv4 world
July 20, 2004
IPv6: DoD Pilot - DREN
19
Overcoming the security issue
(workaround)
• Use DRENv6 testbed for transit to Internet
– use to peer with rest of IPv6 enable Internet and other testbeds
– continue to operate as an “untrusted” IPv6 network
• Enable IPv6 on new DREN2 (MCI) production network.
– Dual stack everywhere.
• Establish trusted gateways between v6 enabled DREN2 and the
DRENv6 testbed
– Upgrade HPC Network Intrusion Detection Systems (NIDS) to be
v6-compliant, monitored by the HPC Computer Emergency
Response Team (CERT), and install at the trusted gateways.
– Install v6 version of standard DREN v4 Access Control Lists (ACLs)
to protect pilot network to same level as IPv4 production network.
• DREN customers receive “safe” native IPv6 service via existing
service delivery point (SDP), in parallel with IPv4 service.
July 20, 2004
IPv6: DoD Pilot - DREN
20
DREN IPv6 transition architecture – FY04
To 6bone, Abilene, and other IPv6 enabled ISPs
IPv6 demonstrations (Moonv6)
links run native IPv6 where
possible, otherwise
tunnelled in IPv4
DRENv6 (Testbed)
Native IPv6 backbone
SSCSD
ARL-APG
ERDC
Testbed at
DREN site
Testbed at
DREN site
v6 ACL
sdp.sandiego
NIDSv6
v6 ACL
NIDSv6
NIDSv6
v6 ACL
sdp.erdc
DREN2 (Production / Pilot)
sdp.arlapg
Dual stack IPv4 and IPv6 wide area infrastructure
sdp
Goal: As secure as
the IPv4 backbone
July 20, 2004
sdp
sdp
Type “A” (IP) production service to DREN sites
IPv4 and IPv6 provided over the same interface
IPv6: DoD Pilot - DREN
21
Site Security Solution
(Example – SPAWAR)
• SPAWAR Intrusion
Detection System (IDS)
modified to support IPv6
• Netscreen Firewall
operating “beta” release
with IPv6 support in
parallel with production
firewall.
WAN
DREN
2
(Pilot)
IPv4 unicast and
multicast services
+ IPv6 unicast
SPAWAR
Border router
(Juniper M20)
IPv4
IDS
IPv6
Netscreen 500 Netscreen 208
Firewall
Firewall
Note: Netscreen (Juniper) now has
mainstream IPv6 support for some models.
Production
Firewall
switch
IPv6 Firewall
(beta code)
to LAN
July 20, 2004
IPv6: DoD Pilot - DREN
22
Ongoing Security Effort
• Snort 2.0.1
– Upgraded to IPv6 – Ken Renard
– In production use today by HPC CERT
• Snort 2.1.1
– Upgraded to IPv6 and available.
– Unable to get support included in main snort distribution.
• IPSEC interoperability testing in Moonv6 phase II.
• ACL and Firewall testing in next phase of Moonv6
• LIBNIDS
– Work underway to modify for IPv6. Available late summer.
• Kerberos v1.3 (MIT)
– IPv6 updates for DREN release by Ken Hornstein (NRL)
• Working on IPv6 for…
– DoD CAC with OpenSSL, PKI, OCSP, LDAP
July 20, 2004
IPv6: DoD Pilot - DREN
23
Goal #5: Provide product feedback, lessons learned,
published via web
Complete
• DREN IPv6 knowledge
base
– https://kb.v6.dren.net
• Open to all DoD (with
PKI certificate)
– Online and ready for
articles
– Initial articles published
• Challenge: getting
people to input their
lessons learned.
July 20, 2004
IPv6: DoD Pilot - DREN
24
Large projects with interest in
IPv6, using DREN
• Global Information Grid (GIG) related
experiments (NRL, SPAWAR)
• Future Combat System (FCS) (Army)
– Existing DREN sites, plus 8 new Boeing sites
• E10A Constellation (Air Force).
• Fleet global unified routing architecture
(Navy), FORCENET
• Military Service Academies
– Train future leaders to expect benefits of IPv6
July 20, 2004
IPv6: DoD Pilot - DREN
25
Mobility Utilization
– Transition to support future mobile soldiers: Force XXI Land
Warriors
Helmet mounted computer and display systems, weapons with video imaging tied
to GPS, backpacks with satellite and ground communication links, radios, 15
pounds of batteries, and more computers, all networked with other warriors and
nearby
tanks, helicopters, andIPv6:
personnel
carriers
July 20, 2004
DoD Pilot - DREN
26
Mobility Utilization
• Transition to support future mobile Service platforms:
the Command and Control Constellation E-10A
aircraft
A fully connected
array of platform-,
space-, and land-based
sensors that use
common
standards and
communication
protocols to relay
information
automatically via
machine-to-machine
interfaces
July 20, 2004
IPv6: DoD Pilot - DREN
27
Mobility Utilization
• Transition to support future mobile sensor webs:
blue-water and littoral sensor webs for FORCEnet
July 20, 2004
IPv6: DoD Pilot - DREN
28
Backup
July 20, 2004
IPv6: DoD Pilot - DREN
29
DREN performance
measurement tools
• DREN “AMP”
– Active Performance Measurement system
– IPv6 updates – Phil Dykstra
• nuttcp 4.0 (NRL)
– TCP performance tester (client/server)
– IPv6 updates – Rob Scott (NRL)
– ftp://ftp.lcp.nrl.navy.mil/pub/nuttcp
July 20, 2004
IPv6: DoD Pilot - DREN
30
Addressing
•
•
•
•
2001:480::/32
/44 reserved for each SDP
Sites get a /48
All subnets are /64
– No tiny subnets for point-to-points
July 20, 2004
IPv6: DoD Pilot - DREN
31