Download Chapter 13

FIREWALLS & NETWORK SECURITY with
Intrusion Detection and VPNs, 2nd ed.
13
Intrusion Detection and
Prevention Systems
By Whitman, Mattord, & Austin
© 2008 Course Technology
Learning Objectives
 Describe the various technologies that are used to
implement intrusion detection and prevention
 Define honey pots, honey nets, and padded cell
systems
 Describe the technologies used to create honey
pots, honey nets, and padded cell systems
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 2
Intrusion Detection and Prevention
 Intrusion occurs when attacker attempts to gain
entry or disrupt normal operations of information
systems, almost always with intent to do harm
 Intrusion detection consists of procedures and
systems that identify system intrusions
 Intrusion reaction encompasses actions an
organization takes when intrusion is detected
 Intrusion prevention consists of activities that
deter intrusion
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 3
Intrusion Detection and Prevention
(continued)
 Intrusion correction activities finalize restoration
of operations to a normal state and seek to
identify source and method of intrusion to
ensure same type of attack cannot occur again
 Intrusion detection systems (IDSs) work like a
burglar alarm: detect violation, activate alarm
 Intrusion prevention system (IPS) can detect
intrusion and launch an active response
 IDS and IPS systems often coexist
 Intrusion detection/prevention system (IDPS)
describes current anti-intrusion technologies
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 4
IDPS Terminology
 Alert or alarm: indication a system has just been
attacked or is under attack
 Evasion: process by which attacker changes the
format and/or timing of their activities to avoid
being detected by the IDPS
 False attack stimulus: event that triggers alarm
when no actual attack is in progress
 False negative: failure of an IDPS to react to an
actual attack event
 False positive: alert or alarm that occurs in the
absence of an actual attack
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 5
IDPS Terminology (continued)
 Noise: accurate alarm events that do not pose
significant threat to information security
 Site policy: rules and configuration guidelines
governing implementation and operation of
IDPSs within an organization
 Site policy awareness: IDPS’s ability to
dynamically modify its configuration in response
to environmental activity
 True attack stimulus: event that triggers alarms
and causes an IDPS to react as if a real attack
is in progress
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 6
IDPS Terminology (continued)
 Tuning: process of adjusting IDPS to maximize
efficiency in detecting true positives, while
minimizing false positives and false negatives
 Confidence value: value placed upon an IDPS’s
ability to detect/identify certain attacks correctly
 Alarm filtering: running system for a while to
track types of false positives it generates and
then adjusting IDPS alarm classifications
 Alarm clustering and compaction: process of
grouping almost identical alarms occurring at
almost same time into single higher-level alarm
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 7
Why Use an IDPS?
 NIST reasons to acquire and use an IDPS:
– To prevent problem behaviors by increasing the
perceived risk of discovery and punishment
– To detect attacks and other security violations
not prevented by other security measures
– To detect and deal with the preambles to attacks
– To document existing threat to an organization
– To act as quality control for security design and
administration
– To provide useful information about intrusions
that do take place
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 8
Why Use an IDPS? (continued)
 IPS technologies can respond to detected threat
by attempting to prevent it from succeeding
while IDS cannot
 IDPS operational categories:
– Host-based (operates on the hosts themselves)
– Network-based (functions at the network level)
• Wireless
• Network behavior analysis (NBA)
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 9
Why Use an IDPS? (continued)
 Several IPS response techniques:
– Terminate network connection or user session
that is being used for the attack
– Block access to target from offending user
account, IP address, or other attacker attribute
– Block all access to targeted host, service,
application, or other resource
– Change the security environment
– Change the attack’s content
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 10
Network-Based IDPS
 NIDPSs reside on computer or appliance
connected to network segment and monitor
network traffic
 Compare measured activity to known signatures
to determine whether an attack has occurred or
is underway
 Protocol stack verification: NIDPSs look for
invalid data packets
 Application protocol verification: higher-order
protocols (HTTP, FTP, Telnet) are examined for
unexpected packet behavior or improper use
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 11
Network-Based IDPS (continued)
 Some advantages of NIDPSs:
– Good network design and placement of devices
can enable organization to use a few devices to
monitor large network
– Usually passive devices and can be deployed
into existing networks with little or no disruption
to normal network operations
– Not usually susceptible to direct attack and may
not be detectable by attackers
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 12
Network-Based IDPS (continued)
 Some disadvantages of NIDPSs:
– Can become overwhelmed by network volume
and fail to recognize attacks they might otherwise
have detected
– Require access to all traffic to be monitored
– Cannot analyze encrypted packets, making some
of the network traffic invisible to the process
– Cannot reliably ascertain if an attack was
successful or not
– Some forms of attack are not easily discerned,
specifically those involving fragmented packets
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 13
Wireless NIDPS
 Monitors and analyzes wireless network traffic
looking for potential problems with wireless
protocols (Layers 2 and 3 of the OSI model)
 Cannot evaluate and diagnose issues with
higher-layer protocols like TCP and UDP
 Some issues with implementation include:
–
–
–
–
–
Physical security
Sensor range
Access point and wireless switch locations
Wired network connections
Cost
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 14
Network Behavior Analysis System
 Examines network traffic to identify problems
related to flow of traffic
 Uses a version of anomaly detection method
 Typical flow data relevant to intrusion detection
and prevention includes:
– Source and destination IP addresses
– Source and destination TCP or UDP ports or
ICMP types and codes
– Number of packets and bytes transmitted in the
session
– Starting and ending timestamps for the session
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 15
Network Behavior Analysis System
(continued)
 Typically monitors internal networks;
occasionally monitors internal/external network
connections
 Most sensors, passive mode deployment only
 Types of events most commonly detected by
NBA sensors include:
–
–
–
–
–
Denial-of-service (DoS) attacks (including DDoS)
Scanning
Worms
Unexpected application services
Policy violations
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 16
Host-Based IDPS
 Resides on particular computer or server (the
host) and monitors activity only on that system
 Also known as system integrity verifiers
 Benchmark/monitor status of key system files
 Triggers alert when file attributes change, new
files are created, or existing files are deleted
 Managed HIDPSs can monitor multiple
computers simultaneously by creating a
configuration file on each monitored host and by
making each HIDPS report back to a master
console system
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 17
Host-Based IDPS (continued)
 Some advantages of HIDPSs:
– Can detect local events on host systems and
also detect attacks that may elude NIDPSs
– Functions on host system, where encrypted
traffic will have been decrypted and is available
for processing
– Unaffected by use of switched network protocols
– Can detect inconsistencies in how applications
and systems programs were used by examining
records stored in audit logs, enabling it to detect
some types of attacks, including Trojan Horse
programs
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 18
Host-Based IDPS (continued)
 Some disadvantages of HIDPSs:
– Pose more management issues since they are
configured/managed on each monitored host
– Vulnerable to direct attacks, attacks on host OS
– Not optimized to detect multi-host scanning;
unable to detect scanning of non-host devices
– Susceptible to some denial-of-service attacks
– Can use large amounts of disk space to retain
the host OS audit logs
– Inflicted overhead on host systems may reduce
system performance below acceptable levels
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 19
IDPS Detection Methods
 Signature-based (knowledge-based, misusedetection) IDPS: examines network traffic in
search of patterns that match known signatures
 Statistical anomaly-based (stat, behavior-based)
IDPS: compares sampled network activity to
established baseline
 Stateful protocol analysis (SPA) IDPS: uses
profiles to detect anomalous protocol behavior
 Log file monitor (LFM) IDPS: reviews log files
from servers, network devices, and other IDPSs
for signatures indicating an attack or intrusion
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 20
IDPS Response Behavior
 Response depends on organization’s policy,
objectives, and system capabilities
 Responses classified as active or passive
 Active response: definitive action automatically
initiated when certain types of alerts are
triggered; can include collecting additional data,
changing or modifying the environment, and
taking action against the intruders
 Passive response: report information they have
collected and wait for administrator to act
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 21
IDPS Response Behavior (continued)
 Some possible responses IDPSs can produce:
–
–
–
–
–
–
–
–
–
–
Audible/visual alarm
SNMP traps and plug-ins
E-mail message
Page or phone message
Log entry
Evidentiary packet dump
Take action against the intruder
Launch program
Reconfigure firewall
Terminate session or connection
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 22
Selecting IDPS Approaches and
Products
 Technical and policy considerations
– What is your system’s environment?
– What are your security goals and objectives?
– What is your existing security policy?
 Organizational requirements and constraints
– What requirements are levied from outside the
organization?
– What are your organization’s resource
constraints?
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 23
Selecting IDPS Approaches and
Products (continued)
 IDPSs product features and quality
– Is the product sufficiently scalable for your
environment?
– How has the product been tested?
– What is the user level of expertise targeted by
the product?
– Is the product designed to evolve as the
organization grows?
– What are the support provisions for the product?
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 24
Strengths and Limitations of IDPSs
 IDPSs perform the following functions well:
– Monitoring and analysis of system events and
user behaviors
– Testing security states of system configurations
– Baselining security state of system and then
tracking any changes to that baseline
– Recognizing patterns of system events that
correspond to known attacks
– Recognizing patterns of activity that statistically
vary from normal activity
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 25
Strengths and Limitations of IDPSs
(continued)
 More functions that IDPSs perform well:
– Managing operating system audit and logging
mechanisms and the data they generate
– Alerting appropriate staff by appropriate means
when attacks are detected
– Measuring enforcement of security policies
encoded in the analysis engine
– Providing default information security policies
– Allowing non-security experts to perform
important security monitoring functions
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 26
Strengths and Limitations of IDPSs
(continued)
 IDPSs cannot perform the following functions:
– Compensating for weak or missing security
mechanisms in the protection infrastructure
– Instantaneously detecting, reporting, responding
to attack during heavy network/processing load
– Detecting newly published attacks or variants
– Effectively responding to sophisticated attacks
– Automatically investigating attacks
– Resisting all attacks intended to defeat them
– Compensating for fidelity issues of data sources
– Dealing effectively with switched networks
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 27
Deployment and Implementation of an
IDPS
 IDPS control strategies
– Centralized: all IDPS control functions are
implemented and managed in a central location
– Fully distributed: all control functions are applied
at the physical location of each IDPS component
– Partially distributed: combines the best of the
other two strategies; while individual agents still
analyze and respond to local threats, their
reporting to a hierarchical central facility enables
the organization to detect widespread attacks
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 28
Deployment and Implementation of an
IDPS (continued)
 IDPS deployment
– Great care must be made in deciding where to
locate IDPS components, physically and logically
– During deployment, each component should be
installed, configured, fine-tuned, tested, and
monitored
– NIDPS and HIDPS used in tandem can protect
individual systems and organizational networks
– Use a phased implementation strategy so as not
to affect entire organization all at once
– First implement NIDPSs and then install HIDPSs
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 29
Deployment and Implementation of an
IDPS (continued)
 Deploying network-based IDPSs
– NIST recommends four locations for NIDPS
sensors:
•
•
•
•
Behind each external firewall, in the network DMZ
Outside an external firewall
On major network backbones
On critical subnets
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 30
Deployment and Implementation of an
IDPS (continued)
 Deploying host-based IDPSs
– Proper implementation of HIDPSs can be a
painstaking and time-consuming task, as each
HIDPS must be custom configured to its host
– May be beneficial to practice an implementation
on one or more test servers beforehand
– Installation continues until either all systems are
installed or organization reaches the planned
degree of coverage it is willing to live with
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 31
Measuring the Effectiveness of IDPSs
 When selecting an IDPS, one typically looks at
four measures of comparative effectiveness:
–
–
–
–
Thresholds
Blacklists and whitelists
Alert settings
Code viewing and editing
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 32
Measuring the Effectiveness of IDPSs
(continued)
 Once implemented, IDPSs are evaluated using
two dominant metrics:
– Administrators evaluate the number of attacks
detected in a known collection of probes
– Administrators examine the level of use,
commonly measured in megabits per second of
network traffic, at which the IDPSs fail
 In order to truly assess effectiveness of IDPS
systems, test process should be as realistic as
possible in its simulation of actual event
 Couple realistic traffic loads, levels of attacks
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 33
Honey Pots, Honey Nets, and Padded
Cell Systems
 Honey pots (decoys, lures, fly-traps): decoy
systems designed to lure potential attackers
away from critical systems
 Honey net: collection of honey pots connecting
several honey pot systems on a subnet
 Honey pots are designed to:
– Divert an attacker from critical systems
– Collect information about the attacker’s activity
– Encourage the attacker to stay on the system
long enough for administrators to document the
event and, perhaps, respond
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 34
Honey Pots, Honey Nets, and Padded
Cell Systems (continued)
 Padded cell: honey pot that has been protected
so it cannot be easily compromised—in other
words, a hardened honey pot
 In addition to attracting attackers with tempting
data, padded cell operates in tandem with
traditional IDPS
 When IDPS detects attackers, it seamlessly
transfers them to special simulated environment
where they can cause no harm
 Allows organization to observe and document
actions and tactics of an attacker
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 35
Honey Pots, Honey Nets, and Padded
Cell Systems (continued)
 Advantages of using honey pot or padded cell:
– Attackers can be diverted to targets that they
cannot damage
– Administrators have time to decide how to
respond to an attacker
– Attackers’ actions can be easily and more
extensively monitored, and the records can be
used to refine threat models and improve system
protections
– Honey pots may be effective at catching insiders
who are snooping around a network
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 36
Honey Pots, Honey Nets, and Padded
Cell Systems (continued)
 Disadvantages of using honey pot or padded
cell:
– The legal implications of using such devices are
not well defined
– Honey pots and padded cells have not yet been
proven as generally useful security technologies
– An expert attacker, once diverted into a decoy
system, may become angry and launch a more
hostile attack against an organization’s systems
– Administrators and security managers need a
high level of expertise to use these systems
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 37
Trap and Trace Systems
 Use a combination of techniques to detect an
intrusion and then to trace it back to its source
 Trap usually consists of a honey pot or padded
cell and an alarm
 Trace feature is process by which organization
attempts to determine identity of an intruder
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 38
Trap and Trace Systems (continued)
 If intruder is someone inside the organization,
administrators are within their power to track the
individual and turn him or her over to authorities
 If intruder is outside security perimeter of the
organization, numerous legal issues arise
 Back hack: hacking into a hacker’s system to
find out as much as possible about the hacker
 Enticement or entrapment?
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 39
Active Intrusion Prevention
 Some organizations do more than wait for an
attack and implement active countermeasures
 When attacker sends ARP request to unused IP
address, LaBrea pretends to be a computer at
that address, allowing attacker to connect
 Once connected, LaBrea changes TCP sliding
window size to a low number to hold open the
connection from the attacker
 This greatly slows down network-based worms
and other attacks and gives LaBrea system time
to notify system and network administrators
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 40
Chapter Summary
 Intrusion occurs when attacker attempts to gain
entry or disrupt normal operations of information
system, almost always with intent to do harm
 Intrusion detection consists of procedures and
systems that identify system intrusions
 Intrusion reaction encompasses actions an
organization takes when intrusion is detected
 Intrusion prevention consists of activities that
deter an intrusion
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 41
Chapter Summary (continued)
 Intrusion detection system (IDS) works like a
burglar alarm: detects violation, activates alarm
 Intrusion prevention system (IPS) can prevent
intrusion from successfully attacking the
organization by means of some active response
 Because these systems often coexist, term
intrusion detection/prevention system (IDPS) is
used to describe current anti-intrusion
technologies
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 42
Chapter Summary (continued)
 IDPSs commonly operate as either network- or
host-based systems
 Network-based IDPS functions at network level
 Host-based IDPS operates on hosts themselves
 Systems that use both approaches are called
hybrid IDPSs
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 43
Chapter Summary (continued)
 IDPSs use variety of detection methods to
monitor and evaluate network traffic
 Three methods dominate: signature-based
approach, statistical-anomaly approach, stateful
protocol analysis approach
 Log file monitor (LFM) IDPS is similar to NIDPS
 Using LFM, system reviews log files generated
by servers, network devices, and other IDPSs,
looking for patterns and signatures that may
indicate an attack or intrusion is in progress or
has already occurred
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 44
Chapter Summary (continued)
 Honey pots: decoy systems designed to lure
potential attackers away from critical systems
 Honey net: collection of honey pots connecting
several honey pot systems on a subnet
 A honey pot is configured in ways that make it
look vulnerable to lure potential attackers into
attacking, thereby revealing themselves
 Trap and trace applications use a combination
of techniques to detect intrusion and then trace
it back to its source
Firewalls & Network Security, 2nd ed. - Chapter 13
Slide 45