Download DDoS, Internet Self-Regulation, and the consequences

DDoS, Internet Self-Regulation,
and the consequences
Closing keynote at EuroBSDcon Belgrade,
September 25, 2016
Gert Döring
Gert Döring?
 programmer and unix hacker since 25+ years
 always fascinated by „remote communication“
 author of mgetty+sendfax
 nowadays working on OpenVPN (packet + os stuff)
 came in contact with „networks“ in 1990 (OSI, UUCP), and
with „IP stuff“ in 1993
 backbone routing, BGP, peering, knowing people, …
 evangelizing IPv6 since 1997
 chair of the RIPE address policy WG
EuroBSDcon 2016
2
why Internet, for me?
 communication with people round the world
 send a mail to an open source guy in Australia, receive
answer in 10 minutes
 work on devices anywhere from anywhere
 „just SSH to the machine and repair what is broken“
 (that used to be „dial in via modem“  )
 „end to end“ style of networking
EuroBSDcon 2016
3
why Internet, for „the 99%“?
 communication with people round the world
 „stay in touch with family members travelling“
 Entertainment („TV with more channels“)
 youtube, netflix, …
 electronic commerce
 shopping, banking, …
 „client to server“ style of networking
EuroBSDcon 2016
4
Internet attacks
 Internet is a mirror of society…
so we have nice folks, rude and stupid folks, criminals, …
 attacks targeting Infrastructure
 hacking, cracking, denial of service, …
 attacks targeting people
 steal identities, banking data, money, …
 different motivations, e.g.: DDoS attacks to extort money –
or as childish revenge for a rude comment on IRC
EuroBSDcon 2016
5
focusing on DDoS attacks
 DoS attack: denial of service
 various means – ping of death, hack PHP forum, distort routing, …
 DDoS attacks: distributed DoS
 high number of participating „attacker machines“
 usually hacked windows zombies (or, more recently, hacked unixes
with lame PHP apps) under attacker control
 „smart“ attacks – „request most expensive search page on web
site 1000 times/second“
 „stupid“ attacks – „just flood site with 20 Gbit/s of garbage
packets, filling uplinks, drowning other traffic“
EuroBSDcon 2016
6
reflective DDoS attacks
 DoS attacks come from „few“ machines – easy to
track, and get shut down (or firewall away)
 DDoS attacks from bot armies are harder, but still
somewhat easy to track: you know who is creating
the evil traffic – so look at flow data, send complaints,
get ISPs to clean up customer badness
 reflective DDoS is the new black: use totally innocent
machines and services to send attack traffic for you
EuroBSDcon 2016
7
reflective DoS explained
blue ISP
more
miscreants
victim server
Internet
green ISP
amplifiers
malicious client
EuroBSDcon 2016
8
reflective DoS explained
blue ISP
victim server
victim and blue ISP
can only see and track
(innocent?) amplifiers,
not real miscreants
more
miscreants
Internet
more queries
with same
faked
source
green ISP
amplifiers
malicious client
EuroBSDcon 2016
9
why is this so great?
 reflective part hides attacking machines:
 to find attacking machines, ISPs along the chain „reflector ->
spoofing source“ need to cooperate to backtrack traffic
 usually not enough visibility or no working contacts
 amplification factor makes attacks (much!) larger
 up to 50x increase in attack volume easy to achieve
(http://www.christian-rossow.de/articles/Amplification_DDoS.php)
 example: 20 hacked machines sending 100 mbit/s each, with
an amplification factor of 50x (using 1000s of amplifiers) ->
100 Gbit/s incoming traffic at victim (20*100*50)
EuroBSDcon 2016
10
this is how it looks like…
 Incoming DDoS attack at
SpaceNet, two weeks ago
 based on NTP reflection, so
easily filterable on router
(ACL), but volume too high
 upstream provider link went
into saturation, customer
had to be taken offline
 diagram shows „normal“
volume on link vs. „attack
volume“ – overprovisioned,
but still „not enough“
EuroBSDcon 2016
11
why is this such a big problem?
 smaller ISPs usually have „multiple 10Gbit/s“ external links
 attacks fill all available bandwidth, hurting all customers
 so ISPs have to resort to null-routing customer under
attack -> DoS succeeded, target is off-line
 „large networks“ (like Akamai or Cloudflare) have seen –
and weathered – attacks over 500 Gbit/s
 so, in the long run, customers will (need to!) migrate to
large service providers, or pay extra for external DDoS
mitigation services that have „lots of bandwidth“
 (potential) long term consequence: market concentration,
small ISPs will get bought or go out of business
EuroBSDcon 2016
12
side track: find human attacker?
 if the attacking machines are hiding behind reflectors
and „uninterested“ network providers, can we find
the attacking human? By tracking motives, e-mails,
money?
 ransom demands are send via hacked PHP scripts or
freemail services, and payment is via Bitcoin
 „do not reply, just pay, we´ll see the money come in“
EuroBSDcon 2016
13
So, let´s shut down these amplifiers!
 Nobody needs open NetBios, Echo or Chargen ports facing the
Internet! Banish the Evil Protocols!
 Nobody needs open NTP servers on the Internet anyway
 Nobody needs open DNS recursors (recursive DNS servers) on the
Internet anyway
 Nobody needs authoritative DNS servers with DNSSEC.
 Wait, what? Uh, ok, let´s mandate rate limiting!!
 This TCP thing is really bad, can be used to amplify small-packet rate –
1x SYN  6-10x SYN/ACK.
 So, let´s rework the whole TCP layer! … wait, what?
EuroBSDcon 2016
14
what is the real problem here?
 real problem is not „servers that answer queries“
but „source IP spoofing“:
 sending IPv4 or IPv6 packets with a source address
that the sender has no authority over, to other parties
outside the sender´s authority
 „not your source“ and
 „not your destination host“
 used in „LAN attacks“ (traffic hijacking etc.) as well
 focus here: WAN, aka „the Internet“
EuroBSDcon 2016
15
what is „the IP spoofing problem“?
 For two-way IP communication, both parties need to send
packets with „their own“ source address, that is, an
address that is routed back to that party
 Under normal circumstances, there is no need to ever send
packets from a source address that would not be routed
back to you
 But it can be nicely used for attacks on others:
 reflective DoS attacks
 TCP stream interference (data injection, resets)
 gaining unauthorized access (the 15+ year old rsh attack)
EuroBSDcon 2016
16
new approach, fix problem at source:
uRPF
 „unicast reverse path filtering“, uRPF
 teach routers to verify source address on ingress
 take incoming packet´s source address
 do a route lookup for the source address
 if the result of the route lookup („where would a packet with that address be sent
to?“) does not point to the interface where it came in: drop packet.
 if verification succeeds, forward normally
 described 14 years ago in RFC2827 / BCP38
 implemented by most vendors
 also known as S.A.V.E. = „source address verification everywhere“
 (nitpick: this is „strict mode“ uRPF. „loose mode“ uRPF = „any route is OK“)
EuroBSDcon 2016
17
uRPF visualized
source ip = blue
source if = client
dst if for ‚blue‘? = client
OK, forward!
benign client
blue isp router
Internet
DROP
amplifiers
malicious client
EuroBSDcon 2016
green isp router
source ip = blue
source if = client
dst if for ‚blue‘? = INET
RPF fail, DROP!
18
uRPF examples
 Cisco:
interface GigEth 3/8
ip verify unicast reverse
 Juniper:
edit interface ge-0/3/0 unit 0 family inet
set rpf-check;
 pf(4):
block in quick from urpf-failed label uRPF
EuroBSDcon 2016
19
so, why is uRPF not universally
deployed?
 It perfectly solves the spoofing problem…
 … for everyone else: you filter, nobody else is attacked by
your customers – you pay, everybody else benefits. So the
commercial incentive is negative.
 peer pressure could help here...
 plus, there are corner cases where it indeed gets in the
way, causing issues for legitimate traffic – quite obviously
for asymmetric traffic
 plus, there can be vendor (hardware) limitations
 (issues described in much more detail in hidden slides)
EuroBSDcon 2016
20
problem and solution found?
 to be effective, S.A.V.E needs to be deployed on all
edges (towards untrustworthy customers)
 if a single hosting provider does not filter his VM
cloud, this is enough to launch quite nasty reflection
attacks
 send with 500 mbit/s
 find reflectors that give you 10x amplification
 5 Gbit/s at victim, from a single bad network
EuroBSDcon 2016
29
commercial incentives?
 commercial incentives for providers are all wrong…
 you need to SPEND money to roll out and monitor source
address filtering, and argue with your customers why they
can´t do what they want
 you LOSE money because they are no longer sending these
500 Mbit/s streams to you, paying for bandwidth
 so, why not just SELL Anti-DDoS services instead, to the
customers on the receiving side of your network?
 shareholder mandate is clear: earn money
EuroBSDcon 2016
30
Internet self regulation
 various initiatives to reach out, and tackle S.A.V.E. from a
„a truly trustworthy ISP would do this!“ reputation-based
angle, like MANRS
 http://manrs.org
 not exactly a major success
 not enough backbone providers on the bandwagon
 less than 0.1% of the 40.000+ networks (ASes) connected
to the Internet have signed MANRS manifesto so far
 no good way to reach those that need to be addressed
EuroBSDcon 2016
31
getting regulators involved?
 In some (one!) countries, this works really nicely
 finnish regulator (FICORA): „wouldn´t it be a good idea to enable
source address validation on all customer ports here in Finland?“
 finnish ISPs: „yes, this sounds reasonable, so we´ll all do it!“


https://www.viestintavirasto.fi/en/steeringandsupervision/actsregulationsdecisions/regulations/regulation67oninformationsecurit
yoftelecommunicationsservices.html
https://www.viestintavirasto.fi/attachments/cert/tietoturvakatsaukset/Cyber_review_Q1_2014_EN.pdf
 about 200 countries connected to the Internet – and most do
not have so competent regulators
 remember: ONE non-filtering hosting provider is sufficient
EuroBSDcon 2016
32
back to protocol fixes?
 NTP – keep service, limit amplification
 „peers & version query“ turned off by default nowadays
 „time query“ packets do not provide amplification
 DNS – introduction of DNS cookies
 RFC7873, in BIND 9.11 (and already in NetBSD)
 require two-way handshake before sending „large“ data
 TCP – never re-send SYN/ACK? (just a wild idea)

https://www.usenix.org/conference/woot14/workshop-program/presentation/kuhrer
 if SYN/ACK is lost, trust TCP client to re-send SYN
EuroBSDcon 2016
33
we will be regulated!
 „The Internet“ is (so far) largely self-regulating, both „the
networks“ and „the protocols“
 RIPE / NANOG / …and IETF
 this works well as long as people care enough, and have
(generally) positive intentions
 mix in international organized crime, and things fall apart –
the Internet is large and decentralized, a very good hiding
place, and not robust against negligence plus malfeasance
 annoy Governments enough, and they will act
EuroBSDcon 2016
34
national Internets
 „communication using public computer networks across
national borders are only allowed if the partner
countries have signed an international treaty requiring
strong anti-abuse mandate and enforcement by local
network operators plus law-enforcement cooperation“
 is that somewhat extreme, and unlikely? yes
 would it work? what would it be like? would users care?
EuroBSDcon 2016
35
will users object?
 „most“ Internet users use server-centric applications
 e-mail: hand to „local provider“ to „send to destination“
 forum, chat, social media
– talking to regionalized server cluster
 entertainment – youtube, netflix, ...
– being delivered by regionalized content node
 e-commerce – talking to local AWS instance
 e-banking – talking to their local bank´s web presence
EuroBSDcon 2016
36
youtube example
(1) user uploads
new content
(2) youtube nodes
synchronize
(3) local nodes
distribute to
local users
EuroBSDcon 2016
content is global, but users never connect
to out-of-region (country) servers
37
facebook example
(1) users upload
new content
(2) facebook nodes
synchronize
(3) local nodes
distribute to
local users
EuroBSDcon 2016
content is global, but users never connect
to out-of-region (country) servers
38
all the big content does this
 users talk to regional nodes (region, country, ISP)
 node selection (usually) by Geo-aware DNS, sending user
to „best“ node for client network
 how content enters network differs slightly
 „youtube style“ – user uploads to local node
 „akamai style“ – content network works as reverse proxy
 content delivery network always de-couples content
origin from content consumers
EuroBSDcon 2016
39
but how would clusters synchronize?
 today, in-cluster communication depends on network
 Akamai is using „the Internet“
 Google/Youtube has its own network links (mostly)
 in such a „nationalized“ scenario, providers of
international offerings would need
 „national“ nodes reachable by users in that country
 private network connections to link nodes
 smaller content providers would have to use large CDNs
EuroBSDcon 2016
40
what about corporate VPNs?
 well, „V“ PN is „virtual“ private networks…
 … and before these, „PN“s existed…
 international service providers offer „private lines“
based on MPLS core today
 might be a bit more costly – or not, if no longer
needing money for anti-DDoS services
EuroBSDcon 2016
41
what about e-mail?
 „classic“ any-to-any e-mail is long dead anyway
 blacklists, SPF, DKIM, DMARC, …
 strange requirements for sending SMTP-over-IPv6
 users wandering off from mailing list to webforums
 most likely a few mega-providers (yahoo, microsoft,
google) would remain, and use their existing CDN
infrastructure to provide global mail services
 smaller providers might pay one of the big ones to use
their „secure and world-wide e-mail!“ Infrastructure
EuroBSDcon 2016
42
would I like that?
 no
 well. maybe. end of trojan and virus attacks to my
parents´ computers? sounds tempting…
EuroBSDcon 2016
43
so, what option will it be?
1) roll out BCP38 / S.A.V.E. everywhere (as in, really!)
2) fix all the protocols to stay well below 0.5x amplification
factor if 1) cannot be done
3) all the content will move to large mega-providers (CDNs
or Anti-DDoS hosters). 200 Gbit/s DDoS attacks will be
the new normal (and you´ll be charged for them), small
ISPs disappear
4) self regulation fails to fix this, Governments will step in…
EuroBSDcon 2016
44
conclusion
 reflective & amplified DDoS attacks are a true danger
for the open and diverse Internet of today
 self-regulation is not working too well
 we need to find a solution, that (re-)enables
 end-to-end communication across the planet
 safe communication, even in unfriendly circumstances
 … and which is not (overly) abused by criminals
EuroBSDcon 2016
45
references
 https://queue.acm.org/detail.cfm?id=2578510
 https://www.usenix.org/conference/woot14/workshopprogram/presentation/kuhrer
 https://www.linkedin.com/pulse/you-part-ddos-problem-barry-greene
 http://www.bcp38.info/
 http://arstechnica.com/security/2016/09/why-the-silencing-ofkrebsonsecurity-opens-a-troubling-chapter-for-the-net/
• http://www.space.net/~gert/16_ddos_and_consequences.pptx
EuroBSDcon 2016
46