Download Bro: A System for Detecting NetwoRk Intruders in Real-Time

Bro: A System for
Detecting Network
Intruders in Real-Time
Presented by
Abdulmajid , Ameen, Yaseen ,mohammed
Lecturer :
DR.Nur Izura
Bro Overview
Bro is an open-source, Unix-based Network Intrusion Detection System
(NIDS) that passively monitors network traffic and looks for suspicious
activity.
Bro was developed by Vern Paxson in the Network Research Group at
Lawrence Berkley National Lab, and by the International Computer Science
Institute.
Bro == Big Brother (“watching and taking care of you” and all your
actions).
Original Goals
High-speed, large volume monitoring:
No packet filter drops (Low packet loss rate)
Would allow intruder to Avoid the NIDS.
Real-time notification
The monitor will be attacked
Mechanism separate from policy:
Bro Features
Rich Application-Layer Analysis
Custom Scripting Language
Pre-written Policy Scripts
Powerful Signature Matching Facility
Network Traffic Analysis
Detection Followed by Action
Snort Compatibility Support (snort2bro)
Target Environments
Bro is intended for use by sites requiring flexible, highly customizable
intrusion detection.
Development of bro is primarily driven by research for intrusion
detection and traffic analysis
Bro is specifically well-suited for scientific environments
•
•
Supports intrusion prevention schemes
High-performance on commodity hardware
•
•
•
Extremely useful in networks with liberal (“default allow”) policies
Runs on Unix-based systems (e.g., Linux, FreeBSD, MacOS)
Open-source (BSD license)
Target Environments(con`t)
Bro is designed for use by Unix experts who place a premium on the ability
to extend an intrusion detection system with new functionality as needed,
which can greatly aid with tracking evolving attacker techniques as well as
inevitable changes to a site's environment and security policy requirements.
Some sites may wish to run a commercial IDS as their front-line of defense,
and then also run Bro as a way to:
•
Verify the results of the commercial IDS / defense-in-depth
•
Attain richer forensics capabilities
•
Provide policy-checking capabilities not facilitated by the commercial IDS
•
Experiment with new approaches and incorporate leading-edge research
Bro Deployment
Structure of the system
•
•
•
Bro is layered into three major components . It uses the now-standard libpcap packet-capture
library developed by LBNL's Network Research Group to filter the packet stream in the
kernel for high performance.
Bro is conceptually divided into an “event engine” that reduces a stream of (filtered)
packets to a stream of higher-level network events.
An interpreter for a specialized language that is used to express a site's security policy.
•
The lower-most layers process the greatest volume of data, and hence must limit the
work performed to a minimum.
•
As we go higher up through the layers, the data stream diminishes, allowing for more
processing per data item.
•
This basic design reflects the need to conserve processing as much as possible, in order
to meet the goals of monitoring high-speed, large volume traffic flows without
dropping packets
Bro Architecture
Policy Interpreter
“Policy Scripts” in Bro Policy language
processes event stream ...
... and takes action!
Event Engine
libpcap
Network
Distills filtered stream into high-level
events.
Kernel filters down high-volume stream
via packet capture library.
Tap network link passively, send up a
copy of all traffic.
Architecture
Network
•
Taps network link passively, sends up a copy
of all network traffic.
10
Architecture
Tcpdump
Filter
Filtered Packet
Stream
libpcap
•
Kernel filters down high-volume stream via
standard libpcap packet capture library.
Packet Stream
Network
11
Architecture
Event
Control
Event
Stream
Event Engine
Tcpdump
Filter
Filtered Packet
Stream
libpcap
•
“Event engine” distills filtered stream into
high-level, policy-neutral events reflecting
underlying network activity
•
E.g., connection_attempt, http_reply, user_logged_in
•
These span a range of semantic levels
•
Currently about 300 different types
Packet Stream
Network
12
Architecture
Policy
Script
Real-time Notification
Record To Disk
Policy Script Interpreter
•
Event
Control
Event
Stream
Event Engine
Tcpdump
Filter
“Policy script” processes event stream,
incorporates:
•
Context from past events
•
Site’s particular policies
Filtered Packet
Stream
libpcap
Packet Stream
Network
13
Architecture
Policy
Script
Real-time Notification
Record To Disk
Policy Script Interpreter
•
Event
Control
Event
Stream
Event Engine
Tcpdump
Filter
“Policy script” processes event stream,
incorporates:
•
Context from past events
•
Site’s particular policies
Filtered Packet
Stream
libpcap
Packet Stream
Network
… and takes action:
Records to disk
Generates alerts via syslog or paging
Sends events to other Bro’s
Executes programs as a form of response
14
Packet Filtering
•
capture via pcap API (libpcap)
Filters expressed in same format as tcpdump(8).Also support off-line traces
port finger or port ftp or (tcp[13] & 7 != 0)
•
Using libpcap gains significant advantages: it isolates Bro from details of the
network link technology (Ethernet, FDDI, SLIP, etc.Network .
it greatly aids in porting Bro to different Unix variants (which also makes it
easier to upgrade to faster hardware as it becomes available). Berkeley
Packet Filter (BPF), found on some Unix, it allows libpcap to bring Bro’s
network filter into the kernel (filters aren’t in userland!).
FreeBSD has a Zero-copy Buffer Extension, which makes it Bro’s OS of
choice!
Event Engine
Written in C++.
Distills filtered stream into high-level, policy-neutral events reflecting
underlying network activity!
E.g., connection_attempt, http_reply, user_logged_in, etc.
Protocols supported: 30 protocols
ARP, IP, ICMP, TCP, UDP
DCE-RPC, DNS, FTP, Finger, Gnutella, HTTP, IRC, Ident, NCP, NFS, NTP,
NetBIOS, POP3, Portmapper, RPC, Rsh, Rlogin, SMB, SMTP, SSH, SSLCiphers,
SunRPC, Telenet
Event Engine, cont.
Those protocol analyzers generate over 300 events!
new_connection(c: connection), new_packet(c: connection, p: pkt_hdr),
connection_established(c: connection), tcp_contents(c: connection, is_orig: bool, seq:
count, contents: string), icmp_echo_reply(c: connection, icmp: icmp_conn, id: count,
seq: count, payload: string), ...
Originally, event engine analyzers were written in C++ Class libraries.
Now BinPAC (Binary Protocol Analyzer Compiler) is used.
binpac generates the C++ code.
Event Engine, cont.
•
When the packets arrive, the Event Engine checks if the headers of these
packets are well formed or not. Packages with not well formed headers are
discarded and events of this problem are created.
•
The packets with well formed headers gets a connection state which can be
connection_attempt,connection_established, connection_rejected or
connection_finished for TCP and for UDP the state is defined as udp_request
or udp_reply.
•
The packets are then collected to their corresponding connection. These
collections use different protocols e.g. HTTP, FTP, SMTP and is therefore
handled by the analyzers specialized for each protocol.
•
Whether or not the connection are handled by any analyzer depends on
which analyzers that are enabled in the system.
Event Engine, cont.
•
The analyzer for each protocol then examines the content of the connection.
•
When this procedure is done, the Event Engine has created high-level events
describing the activity and these are now ready for the Policy Script
Interpreter.
•
As analyzers observe activity, generate events
•
Event = name + typed parameter list
•
Queued for asynchronous execution
•
All events triggered by a given packet executed prior to processing next
packet (Along with timers that expire at that point).
Policy Script Interpreter
•
The Policy Script Interpreter takes the high-level events generated by
the Event Engine and compares these with the policy scripts in the
system.
•
The events are sorted in a FIFO list which means the first that comes
along are the first that is processes.
•
Policy Script Interpreter takes action if it detects any suspicious and
dangerous actions or it discards other events not defined in the policy
scripts.
•
Traffic that seems like attacks but aren’t (false negatives), can be
detected at this point, but if the policy scripts are good, this will be
minimal.
Policy Language
Policy analysis scripts which track network activity written in Bro’s
language
Procedural You write event handlers = functions asynchronously executed
that don’t return a value).
Strongly typed (to detect inconsistencies at compile-time).
Rich in types:
USUAL : bool, int, count, double, string, time, interval, port, addr, hostname, pattern.
•
Domain-specific types, such as addresses, ports, subnets
Aggregates: record, table, set, file.
Real-time communication with other Bro instances and Extensive state
management support (Timers; automatic expiration; persistence
Policy Language
Policy analysis scripts written in Bro’s language:
Procedural (you provide event-handlers that are executed
asynchronously and that don’t return a value).
Strongly typed (to detect inconsistencies at compile-time).
Rich in types:
bool, int, count, double, string, time, interval, port, addr, hostname, pattern.
Aggregates: record, table, set, file.
Flex-like regular expressions built into language (pattern)!
Language, cont.
Operators (+, -, *, /, %, !, &&, ||, ?:, relationals, infix, --, ++).
Static, global & const scoping of variables.
const allowed_services: set[addr, port] = {
•[ftp_serv, [ftp, smtp, 20/tcp]],
•[nttp.lbl.gov, nttp]
•};
•
• Statements (if-then-else, when, return, print, add, delete, and function).
•No for loops! But recursion through functions is allowed.
Notices & Alarms
Convention is to express much detection in terms of “notices”
•
•
•
•
I.e., “I’ve noticed the following activity”
•
Notices do not necessarily convey a problem
•
Distributed scripts detect 78 different types of notices
Scripts provide number of ways to decide when to promote a notice to an
“alarm” These can include syslog, sending email, paging
When you see in a script something like
NOTICE([$note=OutboundTFTP, $conn=u,
$msg=fmt("outbound TFTP: %s -> %s", src, dst)]);it’s just a call to a function named
“NOTICE”, using the language’s mechanism for constructing a record
Note: Entire mechanism is add-on to language, not built in
Sample Bro Policy
•
Using the Bro language, sites can write custom policy scripts to generate alarms on any policy
violation.
•
For example, if a site only allows external http and mail to a small, controlled lists of hosts, they could
do this:
const web_servers = { www.lbl.gov, www.bro-ids.org, };
const mail_servers = { smtp.lbl.gov, smtp2.lbl.gov, };
redef allow_services_to: set[addr, port] += {
[mail_servers, smtp],
[web_servers, http], };
•
Bro can then generate an Alarm or even terminate the connection for policy violations:
if ( service !in allow_services)
NOTICE([$note=SensitiveConnection, $conn=c,]);
if ( inbound && service in terminate_successful_inbound_service )
terminate_connection(c);
Sample Bro Alarms
FTP buffer overflow attack
Bro in Practical Use
•
Primary IDS for LBNL/NERSC since 1996
•
Primary IDS for SC00-03 conferences
•
No specialized hardware needed
•
Low cost allows for multiple deployment
•
Requirements
•
FreeBSD
•
Intel platform
•
Fiber tap
•
Disk space to archive data
Sites using Bro (LBNL)
Use multiple Bros across a 10Gb/s link, where each Bro is only executing a
minimal set of polices.
Operational 247 since 1996. Monitors traffic for suspicious behavior or
policy violations: incoming/outgoing/internal
In conjunction with blocking routers, Bro acts as a dynamic and intelligent
firewall
Blocks access from offending IP addresses
Blocks high risk ports
Blocks known high-risk activity
Terminates connections and/or sends alarms
Locates site policy violations (e.g.: Kazaa and gnutella)
Sites using Bro
•
IDS controls a blocking router
•
IDS blocks dynamically when an intrusion attempt is detected or alerts upon
suspicious activity
•
Router blocks statically like a firewall “Intrusion Prevention”
Use of Bro Within NERSC
ESNet
Multiple Bro Systems
Tapped Traffic
Network
Traffic
Filtering
Border
Router
• Real Time Analysis
• Redundant Backup
• Test Box
• Bulk Traffic Recorder
ACL Insertion
Multiple IDS
Tapped Traffic
Tapped Traffic
NERSC
• Snort
• Bro Heavyweight Protocol Analysis
• Bro GRID / SSL Analysis
• Internal Traffic Bro Monitor
• Wireless Network Bro Monitor
Wireless
Network
November 20, 2003
SC2003, Poenix, AZ
Bro at (NERSC)
•
Tied into a paging system for on-call security person
•
Bro check pointed at set intervals (Reports generated whenever checkpointed)
•
Clears out ‘orphaned’ sessions
•
Allows for offline data analysis
Data archiving
•
•
Maintain traffic data for about 3 months
•
Maintain network
Automatic ACL injection has very low false positive rate
•
At NERSC average about 1 every 6 months
NERSC Network Traffic
3 Week Period
Type of Traffic
Number of
Connections
Overall Percentage
of Traffic
Bulk Data Transfer
666,529
83.73%
Grid Services
74,178
7.19%
Web Related
288,3754
5.30%
Database
620,1730
.27%
Mail
200,484
.04%
System Services
185,272
.04%
116
<.1%
Interactive
Hardware Support

Significant Bro research effort these days focuses on hardware to
accelerate network analysis
•
•
•
•
Note, quite different problem than fast signature matching!
Approach #1: Bro Cluster
•
Set of PCs use inter-Bro coordination for joint analysis
•
Scales to x23 performance for 24 nodes (then flattens)
Approach #2: Shunting
•
Custom (simple) FPGA provides intelligent front-end
•
Per-flow decisions on forward/block/divert through IDS/IPS
•
1 Gbps prototype working, undergoing upgrading
Future: Multicore Bro
•
Will require extensive analysis to understand/tune memory locality & intra-analysis
messaging
Bro Future Directions


Grid related technologies

Ability to detect Grid related protocols

X.509 Certificate Analyzer
•
SSL Analyzer
•
Verify certificates are legitimate
Router Shunting

Primary bottleneck in moving packets into user space
•
Leverage router based hardware filtering to analyze “packets of interest”
•
Proof of concept demo at SC01-03
1)Utilizing Bro and Juniper router
2) Hardware based BPF to filter traffic
Port Mirroring
External
Network
Mirrored Traffic
Juniper
GigE
Interface
Internal
Network
Bro
Filter-based Forwarding
External
Network
Filtered Traffic
Filter
Juniper
Internal
Network
GigE
Interface
Bro
Pros
Wow, where do we begin?
Separation of event generation and event handling (mechanism vs. policy).
Code is/was written for a hostile environment.
Bro’s language (to write policy scripts) is quite tractable.
Resilience in the presence of flooding(Via connection compressor)
An active community (mailing list: [email protected]).
Reaffirms that open-source solution are possible!
Support for plugging in external analyses
Pros : Bro Noteworthy Features

Broccoli = Bro Client Communications Library
•
C interface for external programs to transmit & receive values and events
•


Dynamic adaptation to load
•
Via policy/load-levels.bro

Changes current packet filter to activate more/less analysis
User-level state management
•
•
E.g., syslog reader sends log messages to Bro
Entries in tables, sets automatically expire T sec. after Creation / last read / last write
Advanced detection algorithms (Backdoors: protocols running off-port (obviated by
DPD)
Cons
Encrypted Traffic is Bro’s bane:
Act like a SSL proxy (keep *all* necessary certificates?)
IPv6 ... I think it (finally) made it into development code and
asymmetry .
Pretty complex, script-based system
• Requires understanding of the network
• No GUI, just ASCII logs
Only partially documented
• Lacking resources to fully polish the system
The End
Thank You