Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Piggybacking (Internet access) wikipedia , lookup
Deep packet inspection wikipedia , lookup
Parallel port wikipedia , lookup
Distributed firewall wikipedia , lookup
Computer security wikipedia , lookup
Network tap wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
UniPro protocol stack wikipedia , lookup
Connect. Communicate. Collaborate A Network Security Service for GÉANT2 (and beyond….) Maurizio Molina, DANTE TNC 08, Brugges, 20 th May 2008 Outline • • • • The vision Proof of concept Supporting tools Service Outlook Connect. Communicate. Collaborate The vision: enhance NRENs security Connect. Communicate. Collaborate • NRENs have their CERTs to deal with security • and collaborate with each other – Trusted Introducer – GN2 JRA2 • and DANTE can filter traffic on GN2 if NRENs request it…. ! BUT ! • Can we be more proactive to NREN CERTs exploiting the visibility of the GN2 core? The vision (cont.): enhance NRENs security • • • • Connect. Communicate. Collaborate To spot security anomalies in the GN2 core you need data Good old SNMP? Too coarse! Router Logs? Ok, but need to know what you’re looking for Run a darknet? – It’s not where a core network makes a difference – others already do it • NetFlow? yes, but you need good tools! • Routing data? Only as a complement of NetFlow Proof of concept: what can we see with NetFlow data? Connect. Communicate. Collaborate NfSen, enhanced with self written Anomaly Detection extensions Netflow collected on all peering interfaces 1 / 1,000 Sampling 3k flows/s Bits, Packets or Flows? What to use? Connect. Communicate. Collaborate • Flows/s are more indicative of security incidents • But with fixed thresholds, small interesting peaks will disappear in daily cycles! OK, we’re smart… let’s filter! • It’s an “observer” S Input X2 X1 + + S + - K 1 K 2 Connect. Communicate. Collaborate Forecast error k1=(1-p1)*(1-p2) ; k2= (1-p1)+(1-p2) Choice: p1=p2=0.9 •The “error” is used in control loops • Here we use it to spot a deviation from a baseline Does it help? Not if we stick to volumes (e.g. flows/s) … Connect. Communicate. Collaborate TCP flows (filtered) UDP flows (filtered) Are there other more “security sensitive” features? Connect. Communicate. Collaborate • Recent work on Anomaly Detection suggests focusing on the concentration or dispersion of – Flows per IP source address – Flows per IP destination address – Flows per IP source port – Flows per IP destination port • AKA “IP features entropies” Explanation of IP feature entropy Connect. Communicate. Collaborate fraction of total flows received per IP address fraction of total flows received per IP address 0.25 0.25 0.2 0.2 0.15 0.15 0.1 0.1 0.05 0.05 0 0 1 6 11 16 21 26 IP (ranked) 1 6 11 16 21 26 IP (ranked) Traffic more focused towards a few hosts Normal ni ni The Entropy H is: H ( x) log 2 S i 1 S N H varies between 0 (“one point takes all”) and log2N (uniform distribution) IP feature entropy (simplified) Connect. Communicate. Collaborate fraction of total flows received per IP address fraction of total flows received per IP address 0.25 0.25 0.2 0.2 f=0.81 f=0.6 0.15 0.15 0.1 0.1 0.05 0.05 0 0 1 6 11 16 IP (ranked) Normal 21 26 1 6 11 16 21 26 IP (ranked) Traffic more focused towards a few hosts •Percentage of flows associated to top N src IPs, dst IPs, src ports, dst ports • We tried N = 1, 10, 100, 500 • N=10 was the best choice (anomalies appear more evident) IP features entropies (after “observer” filtering) Connect. Communicate. Collaborate TCP features entropies UDP features entropies 10 days of GN2 traffic Drilling on a TCP peak Drillingdown down -Concentration of DST IPs and DST ports receiving flows -Dispersion of SRC IPs and SRC ports -The “bounce” is due to the filter, and needs a state machine to be correctly interpreted! Connect. Communicate. Collaborate • IRC server in Slovenia, receiving a lot of 60 bytes syn pkts on port 6667, mainly from a /16 Subnetwork of an University in the Netherlands. • Likely a “BotNet war”? Drilling on a UDP peak Drillingdown down Connect. Communicate. Collaborate - Concentration of SRC and DST IPs and SRC ports - Dispersion of DST ports -Observe again the “bounce”! • Portscan of host in CARNET, from 4 hosts, 29 bytes packets And on smaller aggregates? “DWS” DrillingNREN downexample -Concentration of SRC and DST IPs and SRC ports -Dispersion of DST ports -Observe again the “bounce”! Connect. Communicate. Collaborate • A few hours routing shift event (primary to backup access) triggers a lot of “noise” • One MUST be able to correlate feature entropies & traffic shifts! • Other than that, peaks are still very clear! And on smaller aggregates? “NON DWS” NREN example Connect. Communicate. Collaborate • Fewer peaks, but still evident Lessons learnt so far Connect. Communicate. Collaborate • IP features entropies evidence also low volume anomalies, and can give an initial hint on the anomaly type, but: – need a state machine to be interpreted – fully automatic conclusions are difficult – one must not be oblivious of big volume shifts and macroscopic events! • A lot of anomalies are “observable” on DWS connectivity – Good reason for having a security service protecting DWS customers! – But we’ve seen attacks/scans between NRENs as well Moving forward Connect. Communicate. Collaborate • With NfSen and self-written extensions we have enough evidence that: – anomalies are observable in the GÉANT2 core – Novel automatic methodologies for their classifications are applicable • However, we are looking at commercial tools for moving to a service – To reduce effort to engineer / maintain / evolve code – Scalability and tool support is an issue for a service Tools requirements Connect. Communicate. Collaborate • Detection of both low and high volume anomalies – (DoS, DDoS, host and Network scans, worms, phishing sites, etc.) • Automatic classification, collection of evidence • Detection of anomaly entry points, suggestion of ACLs • Give correct indications also in presence of sudden traffic shifts due to routing changing/network outages • Robustness to occasional loss of NetFlow records • Work well also with sampled NetFlow Tools’ comparison Connect. Communicate. Collaborate • Work just started, no conclusion yet • We just report “lesson learnt” so far – on paper analysis of some tools (four in some detail) – Interaction with vendors Tools approaches Connect. Communicate. Collaborate • IP features entropy + volume – Pros: no additional info needed, works with low sampling rate, can catch a wide range of anomalies – Cons: needs drill down after “alert” • Volume + “fingerprints” – Pros: precise, an alert is already “a conclusion” – Cons: won’t catch what you don’t look for • Per host behavioural analysis – Pros: precise – Cons: scalability? robustness to low sampling? Tools common features Connect. Communicate. Collaborate • Require NetFlow on ingress links only • Capable of doing NetFlow v5 and v9 • Require SNMP access to routers to read configuration data Tools distinguishing features Connect. Communicate. Collaborate • BGP processing – create POP to POP (or even prefix to prefix) matrixes – correlate big volume shifts to routing changes • Internal routing (e.g. IS/IS) processing – traffic split of peers on internal (backbone) links • NetFlow collection on multiple points (routing tracing) – But this is not really a plus, rather an additional burden for NOT using routing data! Tools distinguishing features (cont.) Connect. Communicate. Collaborate • different approaches to distinguish “normal” from “not normal” behaviour – Principal Component Analysis – Host type classification & rather complex “scoring” system – moving averages – fixed thresholds Service Outlook Connect. Communicate. Collaborate • Primary recipients: NREN CERTs • Info provided: – security alerts about all types of discovered anomalies – Collected evidence – Suggested mitigation actions – Periodic summary reports • Other recipients: APMs, NREN PC, EU commission – For strategic decisions Acknowledgements Connect. Communicate. Collaborate • Prof. Francesco Donati and Dott.sa Gabriella Caporaletti EICAS automazione S.P.A. for the useful hints on the “observer” design and tuning • Peter Haag from SWITCH for the development of NfSen Connect. Communicate. Collaborate Thank you! – Questions? [email protected]