Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Detection Analytics Chris Calvert, CISSP, CISM – Global Director of Solutions Innovation © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. My Job Is Innovation So I Own The Buzzword Slides (Google Trends Report) 2 The Security Industry Is Not Catching Enough Most Badenterprises Guys remain challenged with missing critical breaches. 229 Days is the median duration of how long breaches were present before discovery in 2013 (M-Trends Report) 3 100% of business networks have traffic going to known malware hosting websites (Cisco 2014 Annual Security Report) Why Is This So Hard? Bad guys know how to stay inside the bell curve. 4 Known: Easier to detect Unknown: Harder to detect • • • • • • • • • • • • Matches a signature Goes to a bad place Works in the clear Unauthorized Use Outside of baseline Within monitored infrastructure New behavior Goes to an approved place Works encrypted Authorized Use Inside of baseline Outside monitored infrastructure The Geography Of Security Detection Has Data flows in many ways – where should we catch and analyze it? Changed Tactical: Streams of Data Endpoint and Network Security Signature & Pattern Based Cyber Defense: Real-time correlation Known Attack Patterns Hunt Team: Long term analytics Unknown Attack Patterns 5 • • Context Data Security Data Enterpris e Data Endpoint protection & logs Attacks easily detected / prevented Operational: Rivers of Data • • SIEM and Platform protection Attacks analyzed & responded to Strategic: Oceans of Data Data Ocean • • Often the missing piece Contains important intelligence All Data Is Not Equal The conventional wisdom of collect everything and figure it out later is WRONG! And expensive… • $collect, $process, $analyze, $store, $manage You should consider the small analytics problems first Collect what matters to solving a real problem – are all these logs useful? 6 Describing the Future of Security Detection Adding Advanced Analytics Existing Basic Context Advanced Context Technical Intelligence • • • • • • Asset, Network Identity Application Flow & DPI Detect 7 Malware Detonation IOC Identification Target Human Intelligence • • Explore Explain Advanced Adhoc Query Advanced Search • • • • Small dataset Basic analysis Indicator lists Pivot search Frontier Understand Emerging Sentiment analysis Motivation Analytical Query Visualization • • • Big Data management Analytical datamart Exploratory data analysis Reporting Scoring Data Mining Machine Learning • • • • • • • • Threat Compliance Risk Fidelity Profiling Clustering, Aggregation Affinity Grouping Classification Other Algorithms Real-time Historical Analysis Statistical Analysis Behavioral • • • • • • • • RT Correlation Log Aggregation LT Correlation Epidemiology Distributed R Standard deviation Depth => Increase in Effectiveness Insider Threat Baselining What Stopped Us From This Kind Of Analysis? 8 Analytics Of The Future Relies On Columnar Retrieval Compression 9 Clustering Distributed Query Find Needles & Understand Haystacks Using… Disciplines of Analytics Classification - context (asset model, etc…) Correlation - real-time (ESM) & historical Clustering – common root cause Affinity Grouping - relationships in data Aggregation - assemble attacker profile Statistical Analysis – reporting & anomalies 10 Visualization Of Big Data – Affinity Group This example reveals a command and control infrastructure Business Statement • Find command and control infrastructure in your enterprise Analytics Statement • • Identify affinity groups Investigate anomalous groupings Findings from Visualization • 11 Hierarchical, highly-resilient C&C infrastructure Anomalous Grouping 1 million events Volume Analyzing The Haystack - aka Reporting 12 Time Visualization Of Big Data – Scatterplot This example reveals a low and slow scan Business Statement • Find sophisticated port scan activity (distributed, randomized) Analytics Statement • Plot multiple months of data on one scatterplot Findings from Visualization • 13 Single multi-week scan from distributed, internal sources indicates advanced attacker Billions of events Visualization Of Big Data – Anomaly Chart This example reveals inappropriate communication (bottom 10 phenomenon) Business Statement • Find servers talking to suspicious hosts outside the network Graph filtered from billions of events Analytics Statement • Plot all suspicious successful communications and review Findings from Visualization • • 14 A host communicated w/ suspicious external website Unique in that no other host in the environment has ever talked to this external website Anomalous Line Example: Challenges in collecting DNS Data 1. Why is DNS important? 200000 • 180000 Security and operations 160000 • 140000 • • Huge quantities of DNS packets can move through your core data centers every day Logging severely impacts performance The right information is not logged at all (ex: DNS replies) Events per second 2. Why is this a hard problem? 120000 100000 80000 60000 40000 20000 0 Routers 15 VPN McAfee ePO Active Web Proxy Directory DNS Our Approach To The DNS Malware Question End-to-end handling of DNS events starts with creating a smaller data set Data Analysis & Visualization Data Acquisition Remediation • Drop normal traffic, collect the rest • Real-time and near-time analysis • Block traffic automatically • Goal: Throw out 99% of events • Novel visualizations • Generate threat intelligence • Integration with ArcSight SIEM workflow in SOCs Goal: Throw out 99% of events 16 What kinds of things can we detect? It turns out we can find more than malware, and that this data set was very useful DNS Analytics Findings 17 • Blacklist Matching • Botnet to Command & Control (Known & Unknown Botnet Activity) • Cloud Platform Abuse • SBC Violations • Data Exfiltration • Research in Progress • Beaconing • Cache Poisoning Attempts Exploratory Data Analysis Analytical Process • • • • Select a question to answer Identify the data that matters Reduce the data to a manageable amount Structure the problem (clean the data, categorize, normalize, articulate) • • Conduct formal analysis (data mining, statistics, machine learning) Conduct exploration / visualization (root cause analyze and remove) • Confirm findings and present results http://h30499.www3.hp.com/t5/HP-Security-Products-Blog/Important-Questions-for-Big-Security-Data/ 18 Hunt Team - The Way To Operationalize Analytics 19 Analytical Talent: A Strong Fingerprint Exists Work in small teams – industry average 10 people Using tools more sophisticated than a spreadsheet is a qualifier Analytics personality? - Tom Davenport • Mindset: #1 intellectually curious more important than any specific skill • Desire to learn • Deep desire for creative assignments • Major in STEM and minor in liberal arts • Rigor and discipline are high • Important work matters to these folks 20 Your Hunt Team Needs A 2-Sided Skill Set Roles and Personas Security Specialist: • The “go to” person to get to the bottom of any major security incidents and would be responsible for actively hunting for indicators of breach • This person understand and researched hyper-current attacker tactics, techniques and procedures Data Scientist: data acquisition, analysis design, data preparation, data analytics, data mining, programming, visualization, interpretation, presentation, administration and managing other analytics professionals Security 21 Data Science They’re in there! Let’s find them. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.