Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Table of Contents Introduction 4 Why You Should Read This Guide 4 Who Is This Guide For? 4 Anatomy of a DDoS Attack 5 How Often Do DDoS Attacks Occur? 5 Who Launches DDoS Attacks and What Is Their Motivation? 6 What Are the Different DDoS Attack Methods and How Do They Affect Your Website? 7 What Is the Financial Impact of a DDoS Attack on Your Business? 8 Choosing the Right DDoS Protection Strategy 9 Risk Assessment 9 Mitigating Network Layer DDoS Attacks Deployment Modes BGP Routing-Based DDoS Protection 10 10 11 How BGP Routing Works 11 Edge Router Monitoring 12 Detecting Application Layer Attacks 14 Case Study: eToro 14 Key Technologies and Capabilities 15 Always-On DDoS Protection 16 Case Study: Mobile Nations 17 DDoS Mitigation Requirements Checklist 18 Maximize Your Level of Preparedness 19 Build a DDoS Response Team 19 Create a DDoS Response Plan 19 Identify Single Points of Failure and Bottlenecks 19 Collaborate with Your ISP 19 Set Optimal DNS TTL 20 DDoS Testing 20 Maintenance Aspects 21 Preparation Checklist 21 2 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Responding to an Attack 22 Early Detection 22 Establishing a War Room 22 Working with Other Teams 22 Marketing, Sales and Customer Management 22 Corporate Communications 23 Legal 23 Post-Attack Steps 24 Process Analysis 24 Attack and Mitigation Analysis 24 DDoS Glossary 25 Appendix — Other Organizational Aspects 28 Dealing with the Media 28 Leveraging Social Media 28 Communicating with Employees 28 Responding to Ransom Notes 28 3 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Introduction Why You Should Read This Guide Distributed denial of service (DDoS) attacks can wreak havoc with network operations teams. DDoS attacks are crafted to saturate and overwhelm network resources until they are rendered unavailable to their intended users. As such, this type of cyber threat “crosses the line” between security and network operations. Network ops teams, which are responsible for ensuring the performance and availability of enterprise applications and services to external users, have a vested interest in protecting their production environment from DDoS attacks. Rather than dealing with daily operations, network and capacity planning, network ops teams that do not have the proper mitigation measures in place may find themselves spending long days and sleepless nights investigating the source of DDoS attacks and trying to stop them. Studies show that it’s not a matter of if your organization is going to be targeted by a DDoS attack, but when. Accordingly, good preparation is essential for making sure your organization is ready to quickly identify and respond to DDoS attacks. Organizations that engage in preemptive DDoS response planning are far more likely to limit potential damage and act in an effective manner than those that try to improvise their way through a DDoS-induced crisis. Who Is This Guide For? This playbook is intended to provide network ops professionals with a practical guide for maximizing DDoS preparedness through the execution of a DDoS response plan. It outlines pragmatic steps and best practices for choosing and setting up the right mitigation solution for your organization, as well as describing the various technologies and deployment modes available. We’ll also cover how to effectively respond to an attack, and how to conduct a thorough post-attack analysis for developing follow-up defense strategies. 4 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Anatomy of a DDoS Attack Let’s start by answering a few key questions regarding DDoS attacks, trends, and technologies, as well as examining how these attacks impact your organization. How Often Do DDoS Attacks Occur? Based on industry reports and what we see in our own network, the frequency and prevalence of DDoS assaults continues to rise. According to Verizon’s 2015 Data Breach Investigations Report, the number of reported DDoS incidents doubled compared to 2014. And based on our own Imperva Incapsula data, these attacks show no signs of abating. According to our Q2 2015 DDoS Global Threat Landscape Report, not only are DDoS attacks larger than ever before, they are also more frequent and longer in duration. The largest network attack mitigated in Q2 2015 was 253 Gbps, while the largest application layer assault amounted to 179,700 requests per second. Normal state Under DNS attack To make matters worse, attackers are more relentless than ever. Once targeted, victims of application layer DDoS attacks are hit once a week on average. Over 20 percent of all network layer attacks last over five days. 5 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Given the relative simplicity and low cost of instigating a DDoS attack, as well as the relative impunity perpetrators enjoy, these disturbing trends are hardly surprising. Booter/Stresser (i.e. DDoS for hire) services that can be ordered online for as little as $10 a pop and free DoS toolkits make it simple for practically anyone to launch an attack. Our statistics show that single-vector attacks associated with botnets-for-hire accounted for more than 40 percent of all network layer attacks. Who Launches DDoS Attacks and What Is Their Motivation? Individuals, businesses, and even nation-states launch DDoS attacks, each with their own particular motivation: • Business competition — DDoS attacks are increasingly being used as a competitive business tool. Some are designed to keep a competitor from doing online business or participating in a significant event such as Cyber Monday (the cyber equivalent of blocking the entrance to your competitor’s store). If your site is down, your services are disrupted and consumers may flock to your competitor. Even a very small amount of downtime or service degradation can end up costing a company many thousands of dollars. Prime examples are gaming/gambling and sports betting sites, which are extremely sensitive to latency since transactions take place in real time. A slight slowdown in site performance usually results in gamers and bettors moving to a competitor’s site to complete their game or place a bet. • Cyber vandalism — Cyber vandals target information infrastructures primarily for the thrill and notoriety associated with bringing down a major website or online service. This is their way of making a statement or leaving their mark on the cybersphere. Botnets, downloadable attack tools, and hijacked servers are the tool of choice for cyber vandals, while readily available botnet-for-hire services leave no online network, application, service, or website immune from danger. Cyber vandals also may employ “script kiddies” to do their malicious coding. • Personal rivalry — A personal grudge or anger can also be the motivation behind DDoS attacks. This is the cyber equivalent of taking the air out of the tires of your boss’ car after getting fired. Using DDoS-for-hire services, it’s simple and cheap to launch an attack that will bring down a rival’s personal router or home computer. This type of personal attack is also quite common in the gaming world, where players launch DDoS attacks against gaming servers to gain a competitive edge or to avoid imminent defeat. • Extortion — An up and rising motivation for DDoS attacks is extortion, by which a cybercriminal sends a ransom note to victims before or after an attack. Several prominent online software companies — including MeetUp, Bitly, Vimeo, and Basecamp – have been on the receiving end of extortion-style attacks. Once a site has been targeted, a ransom (usually in the $300 — $400 range) is demanded in exchange for stopping or not carrying out the attack. • Hacktivism — As the name implies, this type of “hacker” is typically motivated by a political or social cause. Hacktivists use DDoS attacks as a means to express their criticism of everything from governments and politicians, to “big business” and current events. If they disagree with you, your site is going to go down (a.k.a. “tango down”). Anonymous is a well-known example of a hacktivist group. 6 The Imperva Incapsula Network Ops DDoS Playbook • PL AY B O O K Cyber warfare — State-sponsored DDoS attacks are being used to silence government critics and internal opposition, as well as a means to disrupt critical financial, health, and infrastructure services in enemy countries. Unlike conventional warfare, it only takes a small number of DDoS attackers and a minimal investment to inflict substantial punitive damage and register dissent with a government’s actions or policies. What Are the Different DDoS Attack Methods and How Do They Affect Your Website? DDoS assaults are intended to do just what the name implies — render websites and other online services unavailable to their intended users. Such attacks are generally divided into two categories: • Network layer attacks clog the “pipelines” connecting your network, website, or online service to the Internet and include UDP Flood, SYN Flood, NTP Amplification, DNS Amplification, SSDP Amplification, IP Fragmentation, and more. These are almost always high-capacity DDoS barrages, measured in bits-per-second (bps, commonly Gbps) and packets-per-second (PPS, commonly KPPS/MPPS). While high bitrate attacks aim to consume the target’s upstream bandwidth, high packet-rate attacks target the processing capacity of networking devices. It should be noted that SYN Floods can cause particular issues, as by flooding a target with requests to open new connections, it consumes its entire connection pool. These attacks are almost always executed by botnets. Network saturation is the primary goal, but as the capacity of these attacks will have an effect on most service providers, they can also cause severe operational damages such as account suspension and massive overage charges. DNS amplification attacks, are an example of network layer attacks. In such an attack, the attacker spoofs the source address and uses the target’s IP by sending a small, specially crafted DNS query to an “open” DNS server, which responds with a large reply (x200 larger than the query) to the spoofed IP — the target. Unless mitigated, the attack will result in network saturation, causing denial of service for legitimate users. Largest network layer attack in Q2 2015, pealing at over 250 Gbps (shown in Zabbix) 7 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K • Application layer attacks seek to overload the resources upon which an application is running by sending a large number of requests that require resource-intensive handling and processing. Also known as Layer 7 attacks and measured in Requests per Second (RPS), this category includes HTTP floods, slow attacks (Slowloris, RUDY), DNS Query Flood attacks, and those targeting vulnerabilities in operation systems, web applications, and communication protocols. This causes high CPU and memory usage that result in increased latency, eventually hanging or crashing the application or operating system completely. Layer 7 penetrations typically mimic legitimate user traffic so as to evade an organization’s common security measures (including network layer anti-DDoS solutions). They do not require high volumes, for even a rate of 50 — 100 requests/second is enough to cripple most mid-sized websites. • Multi-vector attacks Many DDoS attacks consist of long, complex, multi-staged assaults that resemble advanced persistent threats (APT). These employ different methods and can last days, weeks, and even months at a time. While DDoS assaults do not attempt to breach your security perimeter per se, they are often used to smokescreen other malicious activities or to take down security appliances (e.g. web application firewalls) that can lead to compromised servers and data breaches. What Is the Financial Impact of a DDoS Attack on Your Business? Denial of service attacks often last for days, weeks and even months at a time, which makes them extremely destructive to any online organization. They can cause loss of revenues, erode consumer trust, force businesses to spend fortunes in compensations, and suffer long-term reputation damage. As shown by our 2014 DDoS Impact Survey, every hour of an unmitigated DDoS attack costs organizations an average of $40,000. The cost and probability of a specific company getting hit depends on a number of factors, including the organization size, industry, and type of preventive measures in place. Today, with a substantial percentage of attacks lasting for days, and half of all targets being repeatedly hit, a worst-case scenario entails losses of hundreds of thousands — if not millionsof dollars. Percentage of customers affected Collateral Damage from DDoS Attacks 60% 40% 20% 0% Had to Replace Hardware or Software Had a Virus or Malware Installed Experienced Loss of Customer Trust Theft of Customers Data Loss of Intellectual Property 8 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Choosing the Right DDoS Protection Strategy In the real world there’s no such thing as 100 percent prevention. Cybercriminals are going to continue to launch DDoS attacks and some of them are going to hit their targets, regardless of the defenses in place. What you can do to minimize the damage is to prepare your organization in advance to quickly identify and respond to DDoS attacks. This starts with risk assessment and building a DDoS protection strategy aligned with your company’s business needs. Risk Assessment The first step in preparing your organization to deal with a DDoS incident is to understand the scope of your risk. Important basic questions include: • Which infrastructure assets need protection? • What are the “soft spots” or single points of failure? • What is required to take them down? • How and when will I know I’m targeted? Will it be too late? • What are the impacts (financial and otherwise) of an extended outage? The impact of an extended outage due to a DDoS incident can be measured in terms of lost revenue and resources required to recover an asset. This risk needs to be evaluated against the cost of implementing DDoS protection for the asset. With this information in mind, it’s time to prioritize your concerns and examine various mitigation options within the framework of your security budget. Potential DDoS Targets 9 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K As this playbook is intended to address the needs of network ops teams, we have chosen to focus primarily on a strategy for mitigating network layer DDoS attacks that impact core infrastructure services, including web servers, email servers, FTP servers, and back office CRM or ERP platforms. Mitigating Network Layer DDoS Attacks Deployment Modes What follows is a brief description of the different methods for deploying your DDoS mitigation solution: • Border Gateway Protocol (BGP) Routing BGP routing-based solutions are the most common and effective way to protect multiple service types and protocols across an entire subnet range of IP addresses (known as a /24 or C-class subnet). This type of solution is ideal for thwarting large volumetric and advanced DDoS assaults targeting any type of protocol or infrastructure — including HTTP/S, SMTP, FTP, VoIP, et al. This deployment mode also provides origin protection against direct-to-IP attacks (i.e. attacks against network infrastructure/servers that target a specific IP address). While BGP routing is typically provided as an on-demand service, “always-on” BGP routing is an option offered by some DDoS mitigation providers. Besides the fact that not every company owns an entire C-class, a minor drawback to using the BGP routing-based approach is that latency may increase during attacks. This happens because traffic must first be routed through the scrubbing network for cleansing (in the absence of CDN technology to counteract the extra travel distance the data incurs). • Dedicated IP For smaller organizations wishing to protect multiple service types and protocols, but without a full C-class IP range, this is similar to IP-based protection. In this deployment mode (and unlike BGP), the protection provider assigns you a “dedicated IP address” from its own IP range. Using this address, all incoming traffic passes through the provider’s network where it is inspected and filtered. A redundant, secure symmetric GRE tunnel is used to forward clean traffic to the origin IP and to return outbound traffic from the application to the users. • Physical Link / Cross-Connect This mode is identical to the BGP routing model previously described, with one exception. Instead of connecting the protection provider’s scrubbing centers to your network via GRE tunneling, a direct physical link—also known as a cross-connect cable—is used. This most often requires that your infrastructure reside in the same data center as your protection provider. By using a direct physical connection, you’re always assured predictable latency and maximum throughput. 10 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K BGP Routing-Based DDoS Protection This section walks you through the setup and implementation of a BGP routing based solution. How BGP Routing Works Let’s say your organization bought a C-class IP range from an RIR (Ripe, Arin, Apnic) and has been hit by a DDoS barrage. The first thing you need is a way to protect your IP addresses from being directly attacked. The most common way to do this is through BGP routing, which is an on-demand DDoS mitigation method that offloads all incoming network layer traffic to the DDoS mitigation provider’s network. Here’s what you need to do to implement BGP routing: Step 1: Set up a GRE tunnel. The first step is setting up a GRE tunnel and ideally should be performed in advance of an attack. This is a virtual tunnel between the customer edge router and the DDoS mitigation provider. Once this infrastructure is established, the BGP routing can be implemented. The actual setup depends on type of router, vendor, version, etc., and is described in the vendor documentation. The diagram illustrates an example of a standard, fully redundant network implementation. Two routers are deployed at the customer edge. Two GRE tunnels are deployed per ISP for purposes of redundancy. While it is good practice to work with at least two ISPs, it should be noted that even if you work with more than two ISPs, this is not enough to protect you from a volumetric DDoS attack. Your ISP links will simply become saturated and fall one after the next unless you implement the DDoS mitigation layer (in this case Incapsula) to avoid this bottleneck and absorb the attack traffic. Each GRE tunnel is connected to a different PoP on the DDoS provider network. This means that if one router goes down or if one of the tunnels goes down, traffic continues to flow, thus ensuring full redundancy. 11 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Step 2: Activate BGP routing over the GRE infrastructure. Upon detecting a DDoS attack, withdraw your BGP announcements for any affected subnet and instruct your DDoS mitigation provider to announce the subnet on your behalf. From that point on, your DDoS mitigation provider acts as the ISP, advertising all protected IP ranges. This results in all traffic being redirected through a network of distributed scrubbing centers. All incoming traffic is inspected and filtered, and clean traffic is securely forwarded to the origin server on the enterprise network via GRE tunneling. Outbound traffic is returned asymmetrically via your upstream provider. Can I leave BGP routing “always on” to defend my infrastructure against DDoS attacks? By definition, the fact that all traffic gets routed through a third party network adds latency and hampers the user experience. Enterprises with time-sensitive applications, such as online trading sites or gaming sites, cannot tolerate any latency. Thus, network ops teams prefer to activate BGP routing only in the case of a DDoS attack to maintain optimal network performance in routine situations. Naturally, when under attack, a certain amount of latency is a small price to pay in order to ensure network availability. Moreover, many organizations are wary of having all their traffic going through a third party network all the time due to dependency-related issues. However, always-on BGP routing is an option offered by some DDoS mitigation providers. Edge Router Monitoring Currently, most DDoS attack detection activities are still done manually by operators in the NOC. Due to the fact that humans are fallible and DDoS detection is required 24x7, this type of manual network monitoringis neither efficient nor reliable. When BGP routing is deployed as an on-demand service, time-to-mitigation depends on detecting DDoS attacks before they affect your network performance. For this reason, some DDoS mitigation vendors offer edge router monitoring services that complement on-demand infrastructure protection. Such a service alerts network ops teams to DDoS attacks in real time so they can quickly reroute traffic via BGP for mitigation. This external service is backed by an SLA so you don’t have to worry about missing an attack. 12 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Here’s how it works: 1. The monitoring service provider collects and “learns” the client’s network traffic (NetFlow and sFlow statistics) to determine a baseline definition of normal traffic patterns in terms of volumes, file types, IP addresses, and other variables. 2. Network Ops sends a sample of live traffic at pre-defined intervals (e.g. every 10 seconds). The monitoring service analyzes the statistics. 3. The statistics are compared to the baseline using the 95th percentile bandwidth usage calculation. If the service finds an abnormal spike in traffic, file type, etc., it sends an alert. The client determines the level of deviation from the baseline that triggers an alert. 4. Identification and mitigation of DDoS attacks is performed in accordance with the DDoS mitigation provider’s SLA, which defines the duration of time from the moment you’re attacked until mitigation begins. This includes the time it takes to recognize the attack, send an alert, make the BGP announcement to divert incoming traffic to the DDoS mitigation provider network (in some cases this is done by the DDoS Mitigation provider), and actually mitigate the attack. Live traffic monitoring examples: 1. This screenshot shows the total bandwidth consumption for an enterprise under a DDoS attack. The attack peaked at 31.1 Gbps and 40.8 million packets per second. 2. This screenshot drills down into the bandwidth consumption, showing the types of packets being received. As can be seen this example, the vast majority of DDoS traffic was comprised of Large SYN packets. 13 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Case Study: eToro eToro, the world’s leading social investment network, experienced a massive network DDoS attack in July 2014 on a full C-class of IP addresses. The volume of traffic in this attack overpowered eToro’s defenses, and even caused serious connectivity issues with its ISP. As a result of the attack, eToro’s trading systems were taken completely down. Based on the magnitude of this DDoS attack, eToro needed a solution that could be activated for an entire subnet and that was able to safeguard its services against both floods of web traffic and direct-to-IP DDoS attacks. Moreover, as its infrastructure was still “under fire,” it required an anti-DDoS solution that could be onboarded immediately. With these needs in mind, eToro contacted Incapsula about its Infrastructure DDoS Protection service. This on-demand service leverages Border Gateway Protocol (BGP) routing to safeguard critical network infrastructure from volumetric and protocolbased DDoS attacks, such as UDP, SMTP or SYN Floods, executed directly or via DNS/ NTP amplification. The solution protects all core services (web, email, FTP) from DDoS attacks, as well as protecting against direct-to-IP attacks. Working closely with the Incapsula networking team, traffic to eToro’s sites was rerouted from eToro’s ISP to Incapsula scrubbing centers using BGP announcements. Within half an hour, all incoming traffic to eToro’s IP ranges was being routed through Incapsula for inspection and filtering. Legitimate traffic was securely forwarded to eToro’s network using GRE tunneling. Outbound traffic continued to flow normally via eToro’s ISP. Detecting Application Layer Attacks While your network ops responsibilities may not extend to the application itself, this section gives you a basic understanding of the challenges related to application (Layer 7) DDoS attacks. You may not notice these when you’re monitoring, but it’s helpful to understand that low volume attacks may still affect your application or web operations teams. Application layer DDoS attacks are much more difficult to detect than large-scale network attacks. These stealthy assaults are performed by DDoS bots, designed to establish a full three-way TCP connection and to mimic legitimate web traffic (e.g. browsers and other nonmalicious bots). When defending against these stealthy and complex attacks, success does not depend how big you are, but rather how smart your security technology is and how well it can be utilized. 14 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Key Technologies and Capabilities Successful detection of Layer 7 DDoS attacks requires a traffic profiling solution that can scale on demand to accurately profile incoming traffic — i.e. to distinguish between humans, humanlike bots, and hijacked web browsers. You need to be able to detect and filter out malicious bot traffic — without any impact to your legitimate visitors. Accordingly, your traffic profiling solution should cover the following essential detection and mitigation capabilities: • Client Classification Client classification is all about identifying, classifying, and blocking malicious bots with no manual intervention and a low false-positive rate. Client classification lets you identify and filter out these bots by comparing signatures and examining attributes such as IP and ASN info, HTTP headers, cookie support variations, JavaScript footprint and other telltale signs. It also distinguishes between humans and bot traffic, between “good” and “bad” bots, and identifies AJAX and APIs. • IP Reputation IP reputation is another powerful tool that can be used to quickly filter out bad bots. DDoS mitigation services that operate global networks and protect large numbers of customers are positioned to perform wide-scale analysis on automated clients. Once a bad bot is identified, a signature is created for it. All traffic across the network is then screened using that signature. This type of crowdsourcing enables disparate websites across the entire network to actively participate in their own security, thereby benefitting the whole. • Progressive Challenges Progressive challenges are designed to ensure the optimal balance between strong DDoS protection and an uninterrupted user experience. The idea is to minimize false positives by using a set of transparent challenges (e.g. cookie support, JavaScript execution, etc.) to provide pinpoint identification of the client (human or bot, “good,” or “bad”). • Behavior Anomaly Detection Each of the above detection mechanisms can be individually circumvented. That’s why best practices also call for the use of anomaly detection rules to identify possible instances of sophisticated Layer 7 attacks. This layer acts as an automated safety net to catch attacks that may have slipped through the cracks. These rules detect behavioral patterns that are clearly non-human and may indicate hijacked or malware-infected host computers being remotely controlled to carry out a DDoS attack. 15 The Imperva Incapsula Network Ops DDoS Playbook • PL AY B O O K Identifying Web Threats and Malware Use a web application firewall to ensure that your website or application is always protected against any type of applicative hacking attempt (e.g. SQL injection, cross site scripting, illegal resource access, remote file inclusion, and other top 10 OWASP threats.) These traditional attack methods can be used in conjunction with DDoS assaults in multivector attacks. Always-On DDoS Protection If you’re running a commercial website or online application (e.g. SaaS applications, online banking, e-commerce), you’re probably going to want 24x7 always-on protection. In this scenario, DNS redirection can be used to reroute all website traffic (HTTP/HTTPS) through your DDoS protection provider’s network (usually integrated with a CDN). Once traffic enters the provider’s network, various inspection layers identify and filter out malicious DDoS traffic while legitimate traffic continues to flow unhindered to your protected websites. DNS redirection allows for fast and easy onboarding, since it doesn’t require additional hardware or software and lets you keep your existing hosting and application infrastructures. 16 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Case Study: Mobile Nations Mobile Nations uses Incapsula on 35 sites, serving over 42 million mobile enthusiasts every month. Not surprisingly, its “high profile” sites have been targeted by a number of DDoS attacks over the past few months. In a recent instance, one of its core sites was hit by a Large SYN flood attack, which reached 35 Gigabits per second. With always-on DDoS Protection in place, Mobile Nations was informed by the security team at Incapsula of the attack after the fact. Its users were never aware of the attack and its business operations were not affected. Website performance is critical for Mobile Nations’ e-commerce sites, as even the slightest delay can be the difference between completing an online transaction, or losing the consumer’s business altogether. Since Incapsula DDoS Protection is built on top of a global CDN, using this service has also helped to accelerate page load times by optimizing all content delivery. Mitigating Against DNS Servers Deployed as an always-on service, proxy solutions can be used to safeguard DNS servers from targeted DDoS attacks. To set this up, a proxy is deployed in front of your protected DNS servers, where it inspects all incoming DNS requests. It filters out malicious requests, ensuring that only safe queries reach your origin DNS server. Additionally, it also blocks attempts to use your server as a platform for DNS amplification attacks targeting other servers. Depending on the TTL settings of your name server, implementing a DNS proxy solution can potentially be accomplished in minutes (but could take as long as 24 hours). Once enabled, the proxy becomes your authoritative DNS server, while you continue to manage your DNS zone files outside of the proxy network. If you use an external DNS provider, a proxy service can help you avoid huge bills by offloading large volumes of malicious traffic sent to the DNS server. Moreover, it reduces the chances of being blacklisted from their service due to DDoS attacks originating from your site. DNS proxies offer an added benefit in that they can also function as caching servers. If the proxies are deployed globally, such as on a CDN, they can cache DNS requests and return results locally — thereby accelerating DNS server response times. 17 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K DDoS Mitigation Requirements Checklist RE QUIRE ME NTS Conduct risk assessment DE TAILS • Infrastructure assets, applications, and websites Single points of failure Impact of an extended outage Choose deployment mode • Does the solution deployment model make sense for my architecture? Mitigating network layer attacks • • • - BGP routing for infrastructure protection - Dedicated IP if you don’t have a full C-class - Physical link for infrastructure protection in shared data center - DNS redirection for web applications • • Mitigating application layer attacks • • • • Mitigating attacks against DNS servers Always-On vs. OnDemand • • • Does the solution scale on demand to mitigate massive network/protocol layer attacks? Does the solution prevent IP addresses from being directly attacked? Does the solution support edge router monitoring to reduce time to mitigation? What user classification technologies are in place? Can it distinguish between legitimate users and bots? Do the solutions I’m evaluating include a WAF? Does the solution include IP reputation and behavior anomaly detection? How does the solution include a DNS proxy to inspect incoming DNS requests? Will I always be protected by the solution? Do I need to engage it each time an attack occurs? 18 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Maximize Your Level of Preparedness Build a DDoS Response Team Establishing your DDoS response team is a crucial preparatory step toward reducing the impact of a DDoS attack. The first step is to identify the various people and departments within your organization who will be in charge of both planning and execution. Your team must fulfill a range of tasks — from identifying and mitigating an attack to coordinating with ISPs, notifying customers, communicating with the press, and minimizing potential reputation and liability issues. Ideally, your DDoS response team should include representatives from network operations, marketing and sales, customer service/support, legal, and IT security. These stakeholders should collaborate in developing your plan and establishing the roles/responsibilities of each team member — both in terms of planning and execution. Create a DDoS Response Plan The purpose of your response plan is to define various resources, tools, and procedures required to minimize the risk and costs of a DDoS incident before it happens. It should include topics such as identifying points of failure and bottlenecks, organizational roles and responsibilities, mitigation strategies, monitoring, attack recovery, communications planning, and more. These are covered in the following sections. Identify Single Points of Failure and Bottlenecks Your risk assessment process should include identification of single points of failure or bottlenecks that in the event of a DDoS attack could affect your network's availability. For example, today many DDoS attacks are targeted against DNS servers — often an Achilles’ heel of network security. Even if your online systems are protected, a successful attack against your DNS server can render it unavailable; protecting it is critical. You also need to be aware that if you get hit by a DDoS attack larger than the bandwidth capacity from your ISP, it doesn't matter how redundant your configuration is — your pipe is going to get saturated and your network will go down. Consider system redundancy and disaster recovery options that can help you get back online quickly in the event of a prolonged barrage. Collaborate with Your ISP It’s important to clearly communicate with your Internet service provider (ISP) as part of your DDoS response preparation. In large network attacks that can completely strangle your bandwidth, your ISP has no choice but to intervene. Tier 2 and Tier 3 ISPs, in particular, do not always have the bandwidth capacity to absorb large volumetric attacks, which also can result in service degradation for their other customers. "Troublemakers" targeted by DDoS attacks will simply be dropped or their traffic will be null routed by the ISP due to the collateral damage to other customers. Following attack 19 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K suppression, it can require the adoption of a DDoS mitigation service as a condition for the provisioning of future services to your organization. Many ISPs already offer such a service to their customers. In such a case, be sure you understand its options for defending against DDoS attacks. Additionally, confirm your understanding of SLAs regarding response times. In this regard, here are some helpful questions to ask your ISP: • What type of DDoS protection does it offer? • What type of DDoS attacks is it able to protect against (e.g. network layer, application layer)? • What type of assets can it protect: DNS Servers? Infrastructure? Websites? • How much protection does it provide? • What is its SLA in relation to mitigation time? • Can it terminate service to your organization due to a DDoS attack? Set Optimal DNS TTL Time to live (TTL) is the value determining how long a piece of data is valid. In the DNS world, TTL limits how long your current DNS settings are cached with ISPs. This means that if your website’s TTL is set at three hours, other DNS servers won’t bother checking for a DNS update for your domain over that duration. Shorter TTLs can cause heavier loads on name servers because the DNS records must be updated more frequently, however they allow for DNS changes to be propagated more rapidly. If you’re using an on-demand, DNS-based DDoS mitigation solution, your TTL needs to be lowered prior to experiencing a DDoS attack. A low TTL equates to a faster reaction; this is the time it takes to get traffic routed through your solution. For example, if your TTL is set at three hours, then time-to-mitigation is the time it takes you to notice the attack plus three hours for TTL. DDoS Testing Test the effectiveness of your DDoS mitigation service periodically. Particularly if you are using an on-demand solution, such as BGP routing, you don’t want to wait for an actual attack to discover whether everything is in working order. Verify that all relevant parties understand how the mitigation is deployed (and in case of on-demand — how and how quickly), check that settings are tweaked to suit your system, your systems and applications continue to function properly, traffic continues to arrive, and that there is no negative impact on your users. For testing purposes, it is recommended to turn on your DDoS mitigation measures for a twohour period every 3 — 4 months (once a year at an absolute minimum). Certify your systems and applications continue to function properly, traffic continues to arrive, and there is no 20 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K negative impact on your users. Some DDoS mitigation providers bill on a per-incident fee. You may want to contact your provider prior to testing to ensure that you won’t incur undue fees. Also consider using third party DDoS testing (i.e. pentesting) to simulate an attack against your IT infrastructure so you are prepared when the moment of truth arrives. You can test against a wide variety of attacks — not just those you are familiar with. Maintenance Aspects Five years ago, switching IP addresses was a fairly common, short-term method for avoiding DDoS attacks. Today this method is no longer effective, as massive network attacks often target an entire IP range (a.k.a. a subnet). Since the impact on your ISP remains the same, you still run the risk of being kicked off its service. Moreover, today’s DDoS attacks are DNS-aware. Even if your new IP address belongs to a different ISP, the attack is still able to reach its target destination. Switching ISPs works as long as your secondary ISP is being protected from the attack. This means that its anti-DDoS service is already in place and your new IP address is hidden. Regarding network components, if you’re considering upgrading to more robust equipment to deal with DDoS attacks, think again. Your bandwidth is finite, but the size of DDoS attacks continue to grow. Even equipped with a 20 Gbps anti-DDoS appliance in front of your router/ firewall, assaults exceeding that limit will get stopped upstream by the size of your Internet link, creating a problem for both you and your ISP. Preparation Checklist S TE P ACTIVITY DE TAI LS/TIMETABLE 1 Build DDoS response team • 2 Create DDoS response plan • • • 3 Identify single points of failure and bottlenecks • • • • • 4 Coordinate with your ISP • • • • • 5 Optimize DNS Time-to-Live (TTL) 6 Test DDoS readiness • • Identify people and departments that need to be involved Define roles and responsibilities Define resources, tools, and procedures required to minimize the risk and costs of a DDoS incident Plan should cover the steps below DNS server Bandwidth (Internet link size) Router and switches Firewalls and other network equipment Redundancy and disaster recovery options What type of DDoS protection does it offer? What type of DDoS attacks can it protect against (e.g. network layer, application layer)? What type of assets can it protect: DNS servers? Infrastructure? Websites? How much protection does it provide? What is its SLA in terms of time to mitigation? Optimize your DNS TTLs for the type of DDoS solution you choose to deploy Once every 3 — 4 months 21 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Responding to an Attack Early Detection Early detection plays a pivotal role in minimizing the impact of a DDoS assault. Even before bringing down your networks or systems, frontline appliances are affected, attack volume increases, and performance further degrades each second a penetration goes unnoticed. Don’t rely on manual monitoring to get the job done. For best results, we recommend coupling automatic edge router monitoring with instant triggering of mitigation measures to achieve 24×7 DDoS mitigation while eliminating time-consuming manual procedures. Monitor your network and application traffic to look for early warning signals that may indicate a DDoS attack, such as spikes in traffic or abnormal volumes of traffic from a particular country or IP address. Attackers often perform dry runs as a way of assessing their target’s ability to defend against a particular type of attack. Detecting these limited-scope attacks can help you prepare for the onslaught to follow. In addition, keep an eye on social media (particularly Twitter) and public waste-bins like Pastebin.com to discover online buzz that may offer hints that your organization is being targeted for an attack. Establishing a War Room Designate a “war room” to serve as a planning and communications center during an attack. This could be an existing security or network operations center— perhaps even a conference room. Here your response team can review security updates and strategize defense schemes. Assign a lead who will be responsible for all high-level security decisions during the onslaught. Important: Your organization’s email may not be available during this time. Verify that your response plan documents, team contact information (and other key personnel), as well as that of your ISP and DNS providers, is kept in a secure location independent of Internet access. A hard copy of all of this information may be essential. Working with Other Teams The impact of DDoS attacks goes well beyond the network ops team. It’s not enough to put out the fire. To minimize the impact and alleviate potential damage, you’re going to need additional resources and assistance from your colleagues. As mentioned above, having a cross-departmental DDoS response team in place is a key preparatory step. Beyond detecting and investigating a compromise, this team is responsible for notifying customers, maintaining contact with the media, minimizing brand damage and liability issues. Marketing, Sales and Customer Management Maintaining good faith with customers is paramount. Consumers are generally supportive of a company organization under attack; trying to hide it may shift consumer anger from the perpetrator to your business. Marketing, sales, and customer management teams should establish a process for notifying customers and other affected parties that who follow regulations as well as corporate objectives. 22 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Providers of services to other businesses (B2B), in particular, should decide how transparent you need to be when disclosing the details of a DDoS attack, since this information could also impact your clients’ customers. You may want to prepare financial compensation to customers in advance. This includes making plans for potential discounts and service credits, as well as having your call center and customer outreach teams on call following a service outage. Corporate Communications Communicating with media, partners, and the general public soon after a DDoS attack is vital for preserving your organization’s reputation. The public will know that your site, service, or other systems are down — keeping it secret simply fuels fears. Instead, it’s better to explain to customers the difference between a DDoS assault and other types of cyber attacks that place customer data at risk. A communications plan helps your organization minimize brand damage and reduce the financial impact of a DDoS attack, while also preparing it in advance to answer questions from customers, the press, and shareholders (as applicable). Legal There are few, if any, government-mandated requirements for DDoS mitigation or incident reporting. This is partly due to the relative newness of such multi-vector assaults. It can also be attributed to the fact that DDoS attacks typically don’t fall under established areas of regulation in relation to data breaches. This could be changing, however. Given the prevalence of cyber attacks (including a number of high-profile DDoS attacks) in recent years on financial institutions and other businesses, regulators and investors are focusing an increasing amount of attention toward cyber security risk disclosures. The U.S. Securities and Exchange Commission (SEC) already requires corporations to disclose to investors the cyber security risks they face, just as they disclose other material operational risk. 23 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Post-Attack Steps Following a DDoS incident, there is more to do than simply cleaning up and returning to business as usual. Take the time to review the lessons learned and make adjustments where necessary. Process Analysis By analyzing gaps in your DDoS response plan execution from both a technical and business standpoint, you can adjust it to improve execution during future incidents. Here are some items to evaluate: • Consider those preparation steps you could have taken to respond to the incident faster or more effectively. • Adjust assumptions that affected the decisions made during DDoS incident preparation (if necessary). • Assess the effectiveness of your DDoS response process in relation to communications. • Consider what relationships inside and outside your organizations could help you with future incidents. Attack and Mitigation Analysis As part of the postmortem, review the impact of the intrusion in order to evaluate the effectiveness of your DDoS mitigation solution. Use your network monitoring tools to examine the performance of your equipment during the attack. For example, some routers are more sensitive to certain packet types (such as those used in SYN flood attacks) than others. Also be sure to examine alert logs from your security information and event management system. • What type of DDoS attack targeted you? Was it volumetic, application layer, or something else? What was its size and duration? • Which equipment helped you mitigate, even it was only partially successful? • Which attack traffic had the most impact and why? • Which systems suffered the most? The behavior of your network under attack will help you decide whether you need to upgrade equipment and/or switch to a different DDoS protection service. It will also help you focus your protection (or redundancy) on the systems that need it most. 24 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K DDoS Glossary Application DDoS Attacks These attacks seek to overload resources upon which an application is running, for example, by making excessive log-in, database-lookup, or search requests. This type of attack typically mimics legitimate user traffic so as to evade an organization’s common security measures (including network layer anti-DDoS solutions). Also known as Layer 7 attacks. BGP (Border Gateway Protocol) BGP is used to make core routing decisions on the Internet and is the protocol used by organizations to exchange routing information. Incapsula uses BGP to enable organizations to redirect network traffic through its scrubbing centers. Bot A web robot, or simply “bot,” is a computer that is under control of a third party. Botnet A botnet is a network of bots (“zombies”) that can be controlled as a single entity by a command and control system. Botnets are used to launch DDoS attacks. DNS The Domain Name System (DNS) is the way that Internet domain names are located and translated into Internet Protocol (IP) addresses. A domain name is a meaningful and easy-toremember “handle” for an Internet address. DNS Amplification (Reflection) By forging a victim’s IP address, an attacker can send small requests to a DNS server and ask it to send the victim a large reply. This allows the attacker to have every request from its botnet amplified as much as 70 times in size, making it much easier to overwhelm the target with small resources. DoS (Denial of Service) DoS is an acronym for denial of service. A DoS attack typically uses one or a few computers to cause an outage on the target. DDoS (Distributed Denial of Service) A distributed denial of service (DDoS) attack uses many computers (often bots) distributed across the Internet in an attempt to consume available resources on the target. DDoS assaults are intended to do just what the name implies — render a server or network resource unavailable to its intended users. 25 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K ICMP (Ping) Flood An ICMP flood overwhelms the target resource with ICMP Echo Request (ping) packets, generally sending packets as fast as possible without waiting for replies. This type of attack can consume both outgoing and incoming bandwidth, since the victim’s servers will often attempt to respond with ICMP Echo Reply packets, causing a significant overall system slowdown. Layer 3 and Layer 4 DDoS Attacks Layer 3 and 4 DDoS attacks are types of volumetric DDoS attacks on a network infrastructure. Layer 3 (OSI model network layer) and Layer 4 (protocol layer) DDoS attacks rely on extremely high volumes (floods) of data to slow down web server performance, consume bandwidth and eventually shut down access for legitimate users. These attack types typically include ICMP, SYN, and UDP floods. Layer 7 DDoS Attack A Layer 7 (OSI model application layer) DDoS attack is an attack structured to overload specific elements of an application server infrastructure. Layer 7 attacks are especially complex, stealthy, and difficult to detect because they resemble legitimate website traffic. Network Layer Attacks This type of DDoS attack clogs the “pipelines” connecting your network, website, or online service to the Internet. They send huge amounts of traffic, overwhelming connection capacity until your systems become unavailable. Also known as Layer 3/4 attacks. Scrubbing Centers Scrubbing centers are technical facilities designed for filtering malicious DDoS traffic from inbound traffic streams when mitigating DDoS attacks. Learn more about our high-powered scrubbing centers. Security Operations Center (SOC) A security operations center (SOC) is a centralized venue staffed with IT security experts who monitor and defend enterprise networks and their components. Our 24x7x365 SOC provides customers with proactive response and event management, continuous real-time monitoring, policy tuning, summary attack reports, and 24x7 support. SSL Floods Decrypting SSL traffic on the server side requires 15 times more resources than encrypting the traffic on the client side. SSL floods exploit this asymmetry to overwhelm web servers, which are typically able to handle up to 300 concurrent SSL requests. SYN Flood A SYN flood DDoS attack exploits a known weakness in the TCP connection sequence (i.e. the “three-way handshake”). The client tries to establish a TCP connection with the host server, but 26 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K doesn’t respond to the host server’s request for acknowledgement. The host system continues to wait for acknowledgement for each of the requests, tying up resources until no new connections can be made, and ultimately resulting in denial of service. Tear Drop Attacks (TCP Fragment Flood) A teardrop attack involves sending TCP fragments with overlapping, over-sized payloads to the target machine. When the server attempts to assemble the packet, these mangled packets can cause the server to crash. UDP Flood This type of attack floods random ports on a remote host with numerous UDP packets, causing the host to repeatedly check for the application listening at that port, and (when no application is found) reply with an ICMP Destination Unreachable packet. This process saps host resources, and can ultimately lead to its inaccessibility Volumetric Attacks Volumetric DDoS attacks are also known as floods. DDoS attackers seek to overwhelm the target with excessive data, often using reflection and amplification DDoS techniques. See also Layer 3 and Layer 4 attacks. Web Application Firewall (WAF) A web application firewall controls access to a specific application or service by applying a set of rules to incoming HTTP traffic. A WAF is critical for detecting and preventing stealthy Layer 7 DDoS attacks that mimic regular application traffic. Learn more about our Web Application Firewall. 27 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Appendix — Other Organizational Aspects Like any business initiative, thorough planning across the organization is essential for making the DDoS response process as manageable, painless, and inexpensive as possible. While each organization’s response plan will be slightly different, here are some key elements that all plans should take into account. Dealing with the Media Nominate a single spokesperson for the DDoS response team in advance and prepare that person to deal with the media. This ensures consistent external messages and helps to avoid confusion. Your PR team should also have a blog post already written as part of its crisis communication plan so it can be quickly published in the event of an attack. Given the sensational nature of cyber attacks, you can anticipate that a DDoS attack could carry unwanted publicity along with it. Have a communications plan ready so you know how you are going to notify and respond to any media inquiries if the scale of the attack warrants a response. Leveraging Social Media If your organization’s website has been the target of a DDoS attack, it’s possible your blog may also be out of commission (if it’s hosted on the same server as the attack target). In such a case, social channels such as Twitter can be an effective communications vehicle, helping to limit negative publicity. This serves as another reason to invest in a secondary Internet connection, so as to maintain external communication channels while under attack. Communicating with Employees Communicating with employees is essential for several reasons. First of all, you want to be certain that the network ops team, for example, can reach key decision makers or have the authority to make decisions when a site goes down. Non-IT employees may also be seriously impacted by loss of availability to email and other web-based applications. They need to be informed of the situation and given instructions on backup or offline options until systems are back online. Responding to Ransom Notes According to a 2014 Incapsula survey, 46 percent of DDoS victims received a “ransom note” from their attacker, often prior to the assault. Such messages promise to spare the organization in exchange for money. Perpetrators often ask for a few hundred dollars. Kept intentionally small, the demand is seen as affordable to a small business — or easy to hide in the expense report of a mid-level manager within a larger company. The offenders are playing arbitrage — they easily rent a botnet for $500 and then send out $500 ransom notes to 10 or more companies, calculating that some will pay. 28 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K Paying ransom is not recommended. First, there is no guarantee that the attacker will honor their commitment. If a target is seen as willing to pay, the initial requested amount may be raised. Additionally, once an organization is known to pay, there is no guarantee the perpetrator won’t return — much like organized crime extortion and “protection money” schemes. If you receive a ransom note, Incapsula recommends the following: 1. Do not reply to the note. There is no negotiating with attackers, so responding is pointless. 2. Do not pay the ransom for the reasons outlined above. 3. Alert your response team and try to weather the attack using an effective DDoS mitigation solution. 4. Inform your legal team of the attack and send them a copy of the ransom note. Depending on its length and impact, public companies may decide to disclose the event. 29 The Imperva Incapsula Network Ops DDoS Playbook PL AY B O O K About Imperva Incapsula Imperva Incapsula is a cloud-based application delivery service that protects websites and increases their performance, improving end user experiences and safeguarding web applications and their data from attack. Incapsula includes a web application firewall to thwart hacking attempts, DDoS mitigation to ensure DDoS attacks don’t impact online business assets, a content delivery network to optimize web traffic, and a load balancer to maximize the potential of web environments. WEBSITE DDOS SECURITY PROTECTION Application Delivery LOAD BALANCER CONTENT DELIVERY NETWORK Only Incapsula provides enterprise-grade website security and performance without the need for hardware, software, or specialized expertise. Unlike competitive solutions, Incapsula uses proprietary technologies such as client classification to identify bad bots, and big data analysis of security events to increase accuracy without creating false positives. © 2016, Imperva, Inc. All rights reserved. Imperva, the Imperva logo, SecureSphere, Incapsula, Skyfence, CounterBreach and ThreatRadar are trademarks of Imperva, Inc. and its subsidiaries. All other brand or product names are trademarks or registered trademarks of their respective holders. 30 imperva.com