Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Waging War Against the New Cyberwarrior Tom Longstaff [email protected] CERT Coordination Center Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Sponsored by the U.S. Department of Defense 1 2001 Carnegie Mellon University Incidents Reported to CERT/CC 2001 2002 2 2001 Carnegie Mellon University 52,658 82,094 Vulnerabilities Reported 3 2001 2,437 2002 4,129 2001 Carnegie Mellon University Cyber Strategy Cyber-war is not just simple hacking Sociology of warriors vs. hackers - Morale - Organization - Vigilance vs. assumed invulnerability Motivation of warriors vs. hackers - Accountability vs. anarchy - Delayed vs. immediate gratification - Internal vs. external gratification Preparation of warriors vs. hackers - Training - Intelligence / strategy 4 2001 Carnegie Mellon University Incident Trends 5 2001 Carnegie Mellon University Intruder Technology Intruders use currently available technology to develop new technology coordinate propogate coordinate propogate compromise compromise compromise compromise scan scan scan scan scan 1997 6 1998 2001 Carnegie Mellon University 1999 2000 Information Collection, Analysis and Sharing for Situational Awareness 7 2001 Carnegie Mellon University Overview Challenge statement • Too much data – too little information – not shared Operational Need CERT Vision/Goals Our Approach Project Maturity Wrap up 8 2001 Carnegie Mellon University Data Challenge System & Network Administrators overwhelmed • Data overload • Important data often not collected • Local/parochial focus Poor Network Situational Awareness Network Security Information is not shared • Unconnected “Islands of Information” • Ineffective, non-standard security tools and processes • Non-technical reasons (organizational and liability) • Unwilling to yield autonomy to gain better information Attackers share information more efficiently 9 2001 Carnegie Mellon University Our Vision An operationally flexible system providing: •Clear avenues for exchanging relevant data •Improved local monitoring •Improved cueing methods •Cross organization analytical capabilities •Improved indications and warning •Cross organization situational awareness 10 2001 Carnegie Mellon University Our Goal Collect structured, sanitized, and representative situational awareness data in a standardized format to: • Recognize and respond faster (prior to damage) • Permit collection of focused information on activity and trends • Alert operators for proactive response • Provide tools for sites to manage incident information 11 2001 Carnegie Mellon University Bi-directional Solution Top-down •Collection, organization, and analysis of data from wide, shallow sensors Bottom-up •Federation of data from narrow, deep sensors -Alerts from IDSs and Firewalls -Raw data from sniffers & recorders 12 2001 Carnegie Mellon University Top-Down Approach Similar to the DEW line* – early indication that an attack may be coming facilitated by sensing the entire network Analysis for I&W • Hacking involves reverse engineering: the attacker must probe, examine and determine the “right” approach • Frequently precursors to attacks are buried in the “noise” • Improve our ability to detect attacker behavior in the preattack stages Preventive Analysis • Detect configuration errors * DEW - Distant Early Warning 13 2001 Carnegie Mellon University Top-Down Edge Router Netflow Collector T1 OC3 Internet 100Mb Firewall/Router Real time collection; analysis and alert tools 14 2001 Carnegie Mellon University Intranet Top-Down Collect coarse data • No payload data • Headers Only – Source, Destination IP and ports; protocol; times; traffic volumes (e.g. packets and bytes) • Both inbound and outbound Collect wide data • >95% network coverage • Multiple networks Collect a lot of data • Requires a data center with large computational and storage capacity to facilitate historical analysis • Scalable collection and analysis • Outbound data indicates planted code or insiders 15 2001 Carnegie Mellon University Top-Down - Wide Shallow Sensors Netflow • Originally defined by CISCO but increasingly becoming standard • See what the router sees Records of “flows” created at the router • Assist in routing and in reporting network traffic statistics Consists of flow records aggregated from packets Sent to a collector and aggregated into different information records for varied analysis. 16 2001 Carnegie Mellon University Inbound Slammer Traffic UDP Port 1434 Flows 40000000 35000000 30000000 Flows 25000000 20000000 15000000 10000000 5000000 Hour 1/24:00-1/25:18 17 2001 Carnegie Mellon University 18 16 14 12 10 8 6 4 2 0 22 20 18 16 14 12 10 8 6 4 2 0 0 Slammer: Precursor Detection UDP Port 1434 - Precursor 160000 140000 120000 Flows 100000 80000 Series1 60000 40000 20000 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 Hour 1/24:00 1/25:04 18 2001 Carnegie Mellon University 1 2 3 4 Slammer: Precursor Analysis Focused on hours 6, 7, 8, 13, 14 Identified 3 primary sources, all from a known adversary All 3 used a fixed pattern Identified responders: 2 out of 4 subsequently compromised. 19 2001 Carnegie Mellon University Detecting Scans Detect scans against client network hosts •Higher intensity scans •“Low and slow” scans •Coordinated (distributed) scanning 20 2001 Carnegie Mellon University Low-Packet Filtering Sessions Vs. Time, December 12th-14th 2002 450 400 Low Packet Sessions 350 Records 300 All tcp Sessions 250 200 150 100 50 0 Time Time (30 second bins) 21 2001 Carnegie Mellon University Stealth Tool Detection We are studying extremely slow (“1 packet a day scanner”) traffic on the Internet. As an initial trial, we identified sources sending between 1 and 3 packets of TCP (non-Web) traffic per day into the client’s networks. We applied this to the period September 1-11, finding that 0.00001% of the traffic matched this pattern. Further analysis yielded a fingerprint for one tool. The tool’s profile appears to match Compaq Insight Manager XE on the client network. 22 2001 Carnegie Mellon University Bottom-Up Approach Using data from Commercial Off the Shelf (COTS) security solutions already deployed • e.g., Intrusion Detection Systems, firewalls, system logs, Snort, RealSecure, PIX, IPTables, syslog Custom-developed technology (AirCERT), currently not present in commercial products, to integrate, convert, analyze, and share the data Combination enables analysis of security event data from across administrative domains • Different entities • Different scales: - Subsidiary - Corporation - Sector 23 2001 Carnegie Mellon University Bottom-Up To other subnets…. Firewall/Router AirCERT Collector IDS System 24 2001 Carnegie Mellon University Intranet Sensor (Packet Capture) Web Server Mail Server Bottom-Up Collect data from by security devices (firewalls and intrusion detection devices) • All or part of a packet • Testimonials (e.g., IDS alerts), and associated contextual data Collect widely varied data • Maximize network diversity (e.g., edge vs. transit; many administrative domains) • Maximize sensor diversity (e.g., IDS, firewall) Configurable volume of data • Determined by local site and collaborators • Scalable collection and analysis 25 2001 Carnegie Mellon University Bottom-Up Implementation • Flexible, open-source, standards-based reference implementation of an Internet-scalable threat assessment system Capability consists of components for • Data Collection • Data Sharing 26 2001 Carnegie Mellon University Implementation Edge Router Netflow Collector T1 OC3 Internet 100Mb Firewall/Router Collector Intranet IDS System 27 2001 Carnegie Mellon University Sensor (Packet Capture) Web Server Mail Server What Do You Do With This Data? Predictive numerical and statistical analysis • Calculate long-term trends • Profile traffic – map servers, create baselines • Continual monitoring for attack precursors Traffic Analysis • Routing Anomalies and flaws • Packet/Byte Characteristics Weak general results can drive strong focused analysis Analysis from Top-Down can drive Bottom-Up, and viceversa 28 2001 Carnegie Mellon University What Else Do You Do With This Data? Manage and analyze event data at all points in reporting hierarchy to detect and identify • Compromise with cross-site data • Coordinated, distributed attacks • Slow and stealthy scans • Network attack “fronts” • Multi-site trends - Distinguish between local and global activity – Targeted scans – Vulnerability probes 29 2001 Carnegie Mellon University Integrating Top-Down & Bottom-Up Analysis Augment data collection and configuration at the “leaves” Supplement or verify existing local security analyses and processes Employing cues gained from analysis at the “root”, focus analysis on data previously deemed benign or ignored Verify suggestive top-down and cross-site analysis by the selective analysis of data collected at the “leaves” 30 2001 Carnegie Mellon University ACID Architecture Network Link Snort or Firewall ACID Alert Database Web Server (PHP) Browser Browser Browser (Analyst #1) (Analyst #2) (Analyst #N) ACID can only analyze what is in the Alert Database 31 2001 Carnegie Mellon University Views of Data (grouping) • ACID has no implicit analysis functionality -- only presents the data by - Event (Signature) Classification IP Address Port Flow Time Sensor - Charts grouped by time, IP, classification and ports 32 2001 Carnegie Mellon University - User defined queries Event (Signature) view Unique Alert • Identifies the different type of attacks Reference Signature Number of Number of Src/Dst IP Classification Sensors Total Number of Occurrences from Main, click on number next to ‘Unique Alert’ 33 2001 Carnegie Mellon University First/Last Occurrence Classification view • Identifies the different event classifications Classification Number of Number of Number of Src/Dst IP Events Sensors Total Number of Occurrences First/Last Occurrence From Main, click on the number next to ‘categories’ 34 2001 Carnegie Mellon University Address view • Identifies mostly frequently attacked machines • Identifies network blocks of frequent attackers IP Address Fully Qualified Domain Name Number of Sensors Number of Total Number Unique of all Events Events From Main, click on number after ‘IP’ 35 2001 Carnegie Mellon University Number of times seen in opposite direction Port view • Identifies most commonly targeted services Port Number of Sensors Number of Unique Events Total Number of all Events First/Last Occurrence Number of Src/Dst IP From Main, click on number after ‘Port’ 36 2001 Carnegie Mellon University Flow view • Identifies suspicious events by flow activity Protocol FQDN and IP of Source FQDN and IP of Destination Unique Destination Ports 37 Number of Unique Events From Main, click on number after ‘Unique IP LInks’ 2001 Carnegie Mellon University Total Number of all Events Sensor view • Aggregate statistics on sensor Sensor ID Sensor Name Total Number Number of of all Events Unique Events First/Last Occurrence Number of Src/Dst IP From Main, click on number next to ‘# of Sensors’ 38 2001 Carnegie Mellon University Temporal view Alert Listing • Identifies event chronology Event (Signature) [ Query Seq. Number, Returned by any Sensor ID, Event ID ] 39 2001 Carnegie Mellon University Timestamp Searches or Alert Listing Layer-4 IP Src/Dst IP and Port encapsulated Snapshots protocol Temporal view (2) Graph Alert Detection Time • Graphs number of alerts aggregating on hour, day, or month • Visually represents peak attack periods Time Interval Number of Events occurring in the time interval From Main, click on ‘Graph Alert Detection Time’ 40 2001 Carnegie Mellon University Drill-Down: Individual Alert 41 Click on the ID in any Alert Listing 2001 Carnegie Mellon University Drill-Down: IP Address • Provides statistics on an individual IP address • Links to external registries and tools to gather information about the address Click on the IP address in any Alert Listing 42 2001 Carnegie Mellon University User Interface: Main 43 2001 Carnegie Mellon University User Interface: Navigation Currently Selected Criteria Browsing Buttons Checkbox to select alert 44 2001 Carnegie Mellon University ACID Browser “Back”button Alert Actions Analysis Example: Most Frequently Targeted TCP Services 45 2001 Carnegie Mellon University Project Maturity Top-Down • Highly efficient data partitioning and packing format - Does not rely on a relational database – Packs 90+Gb per day into less than 30Gb • Generic analysis tools written to perform ad-hoc analysis - Processes a day’s worth of data in under 10 minutes - Rapid analytical tool development API • Operational deployment at sponsor site Bottom-Up • Prototype collection infrastructure developed and tested • Active involvement in IETF security standards activity • Pilot testing in progress 46 2001 Carnegie Mellon University Project Maturity: Continuing Efforts Involve more pilot sites Improve analytical capabilities Improve automated configuration Continue standards development efforts Increase collection diversity by supporting additional COTS Persuade vendors to adopt standards Planned Extensions to Netflow Analysis • Enhanced with additional data based on payload but packed into the existing form-factor • Aggregation into session records • Matching aggregated session records into transaction records 47 2001 Carnegie Mellon University Summary Transformational approach to data collection, sharing, analysis and response for Computer Network Defense Provides timely, focused information to operators – providing cues for immediate action Provides tools for local, tailored analysis Provides local, enterprise and Internet Situational Awareness information Levels the playing field 48 2001 Carnegie Mellon University Modeling and Simulation How do we drink from this fire hose? Goal is to use the volume of information to gain a predictive power over our adversaries 49 2001 Carnegie Mellon University Emergent Algorithms New Ideas • Survivability is an emergent property of a system Attack Recognize & Resist • Emergent algorithms are distributed computations that fulfill mission requirements in the absence of central control and global visibility • Local actions + Near-neighbor interactions => Complex global properties Adapt Recover Impact • A new methodology for the design of highly survivable systems and architectures • Ability to produce desired global effects through cooperative local actions distributed throughout a system (“self-stabilizing”) 50 2001 Carnegie Mellon University Current Research Design an emergent algorithm simulation environment and language (“Easel”) to: • Simulate and visualize the effects of specific cyber-attacks, accidents and failures • Create a test-bed for mission-critical systems The nature of complex, unbounded systems Easel is a new computer language designed to simulate complex, unbounded systems. Such systems exhibit the following properties • • • • • • Large numbers of autonomous components Incomplete and imprecise information Limited local knowledge No central control Bounded number of neighbors Competing objectives Such systems are more survivable because of • adaptability • graceful degradation • no critical points of failure • awareness of the local environment 51 2001 Carnegie Mellon University Six explorations in survivability cascade failure in organizations failure propagation through an organizational network network topology generation survivability is a function of topology simple network message routing illustration of a very simple routing algorithm network attackers and defenders attackers compromise and defenders patch epidemic dynamics local contact leads to global infection seismic collapse of a building elastic response of linked beams to seismic shaking 52 2001 Carnegie Mellon University Where can Easel help? Provide independent verification that complex system designs have no serious survivability flaws Analyze scenarios with respect to impact of: • • • • • design assumptions human error incomplete or imprecise information common mode failures single point of failure leading to cascading failure • organized malicious attacks 53 2001 Carnegie Mellon University Dealing with the Threat - Fusion Analysis Efforts Data Collection • AirCERT • Open source correlation Individual Event Analysis Statistical Analysis Modeling and Simulation 54 2001 Carnegie Mellon University What’s Next? Our coordination of information must be commensurate with the enemy’s ability to use this information against us We must create a new world of checks and balances to match the appropriate use of information in the pursuit of malfeasants The key to this revolution is local administration of information while maintaining global coordination 55 2001 Carnegie Mellon University Changes in Intrusion Profile 1988 • exploiting passwords • exploiting known vulnerabilities The definition of “vulnerability” on the Internet is approaching that of the DoD in trusted systems 56 2001 Carnegie Mellon University Today • exploiting passwords • exploiting known vulnerabilities • exploiting protocol flaws • examining source and binary files for new security flaws • abusing anonymous FTP, web servers, email • installing sniffer programs • IP source address spoofing • denial of service attacks • widespread, automated scanning of the Internet • deep vuls in SNMP, SSL, WEP, … Scanning for Victims Today: Wide scale scanners collect information on 100,000s of hosts around the Internet Sniffers now use the same technology as intrusion detection tools Number and complexity of trust relationships in real systems make victim selection easier 57 2001 Carnegie Mellon University Scanning for Victims Tomorrow: Use of data reduction tools and more queryoriented search capability will allow reuse of scan data Inexpensive disk and computation time will encourage the use of cryptography and persistent storage of scan data Scan data becomes a commodity like marketing information 58 2001 Carnegie Mellon University The Future of Probes We’re very likely to see more: • widespread brute-force scanning with little regard for being detected • stealthy probes like SYN and FIN that require packet logging to detect • attempts to hide the origin of the probes through spoofing and decoys • automated vulnerability exploits that probe and compromise in a single step 59 2001 Carnegie Mellon University Typical Intruder Attack Internet Yesterday Intruder 60 scans remote sites to identify targets, then attacks vulnerable or misconfigured hosts 2001 Carnegie Mellon University Distributed Coordinated Attack Internet Today Intruder 61 scans remote sites to identify targets, then attacks vulnerable or misconfigured hosts 2001 Carnegie Mellon University Distributed Coordinated Attack Uses 100s to 1000s of clients (10,000s) Is triggered by a “victim” and “time” command Command channels include IRC, SNMP, ICMP May include dynamic upgrade and be spread by worms Will simultaneously attack the victim from all clients Today used in DoS attacks only 62 2001 Carnegie Mellon University Issues for Responding to DoS Attacks Filtering/detecting this attack is problematic! The intruder’s intent is not always clear in denial of service attacks. The intruder might be • using the DoS attack to hide a real attack • misusing resources to attack someone else • attempting to frame someone else for the attack • disabling a trusted host as part of an intrusion Attacks also frequently involve • IRC abuse • intruders attacking each other • retaliation for securing systems 63 2001 Carnegie Mellon University The Future is Automation Put these together and what do you get? • tools to scan for multiple vulnerabilities • architecture identification tools • widely available exploits • pre-packaged Trojan horse backdoor programs • delivery and recon through active content Bad news! Together, these publicly available tools could be modified to launch wide-spread scans and compromise systems automatically. 64 2001 Carnegie Mellon University Warning Signs of Today We •Tolerate unexpected program behavior •Place little value on software quality •Assemble parts with no clear idea what each part does nor who created it •Spread highly capable and functional components through the hands of the unenlightened 65 2001 Carnegie Mellon University Tom Longstaff’s Predictions for the Next Decade (well, at least the next 3 years) Network crime on the rise Many countries and NGOs preparing information warfare weapons Insiders and planted vulnerabilities control the battlespace Information warfare will be combined with traditional tactics (e.g., Iraq) 66 2001 Carnegie Mellon University