Download Cisco Network Visibility Flow Protocol Specification

 Cisco Network Visibility Flow Protocol
Specification
This document contains the protocol specification for the
Cisco Network Visibility Flow (nvzFlow for short).
This document is property of Cisco Systems, Inc. and is shared
under NDA Table of Contents
Introduction ..................................................................................................................................... 3 Design Goals ................................................................................................................................... 3 Cisco Network Visibility Flow (nvzFlow) Information Elements ................................................. 4 Endpoint Exporter IPFIX / nvzFlow Templates ............................................................................. 6 Endpoint Identity Template (ID 262) .......................................................................................... 6 Interface Info Template (ID 265) ................................................................................................ 6 IPv4 Per-Flow Data Template (ID 263) ...................................................................................... 7 IPv6 Per-Flow Data Template (ID 264) ...................................................................................... 8 Data Model and Correlation IDs ................................................................................................. 9 Implementation Considerations ................................................................................................. 10 Future Considerations ................................................................................................................ 10 References ................................................................................................................................. 11 Introduction This document defines the new IPFIX Informational Elements that Cisco Network Visibility Flow (nvzFlow) adds to IPFIX (RFC 7011-­‐7015). These data types carry information that is useful for providing better visibility of the endpoint in a network. The new information elements are defined using the abstract data types defined by IPFIX (RFC 7012, Section 3.1). Design Goals Cisco Network Visibility Flow, or nvzFlow (pronounced: en-­‐vizzy-­‐flow) is designed to provide greater network visibility of endpoints in a lightweight manner. The protocol extends IPFIX to include a small set of high-­‐value data in order to convey the following: Mary visited Salesforce.com from an unmanaged Windows laptop using a company approved browser, while she was connected on her local Starbucks’ Wi-­‐Fi network. The 5 key visibility categories conveyed by the protocol are: •
•
•
•
•
User Device Application Location Destination In order to be included in the protocol, each data element must meet the following requirements: 1. Convey information directly related to the five key visibility categories shown above. 2. Be clear, obvious and of high value to an IT administrator. 3. Is useful to analytics, while not requiring the use of analytics (raw-­‐data value). 4. Easily obtainable information across a wide variety of OS and device types. 5. Lightweight and portable (minimal network, battery or CPU impact; no DPI required; etc.) Cisco Network Visibility Flow (nvzFlow) Information Elements The following table identifies the new Information Elements in the same format as the standard Information Elements defined in http://www.iana.org/assignments/ipfix/ipfix.xhtml All string values shall be encoded in UTF-­‐8. Element IDs below are shown with and without the Enterprise Bit being set. There are 3 templates as part of the nvzFlow Protocol: Endpoint – Fields that are associated with an Endpoint Device, such as OS Name. Interface – Fields associated with the network interface, such as Adapter Name. Flow – Fields associated with the flow itself, such as L4 Bytes Out. Note that some fields will be present in multiple templates for cross correlation purposes. These are mandatory fields, while all others can be either anonymized (“-­‐“) or not sent at all. •
•
•
Element ID Name Data Type with/without enterprise bit Data Type Semantics Description 45100 / 12332 nvzFlowUDID octetArray identifier 20 byte Unique ID that identifies the endpoint. 45101 / 12333 nvzFlowLoggedInUser string default 45102 / 12334 nvzFlowOSName string default 45103 / 12335 nvzFlowOSVersion string default 45104 / 12336 nvzFlowSystemManufacturer string default 45105 / 12337 45106 / 12338 nvzFlowSystemType nvzFlowProcessAccount string string default default 45107 / 12339 nvzFlowParentProcessAccount string default 45108 / 12340 nvzFlowProcessName string default 45109 / 12341 nvzFlowProcessHash octetArray default 45110 / 12342 nvzFlowParentProcessName string default 45111 / 12343 nvzFlowParentProcessHash octetArray default This is logged in user on the device, in the form Authority\Principle. This is different from userName (id: 371), which is user associated with the flow. Name of the operating system. E.g., Windows, Mac OS X Version of the operating system. E.g., 5.0.2195 or 10.10 Name of the manufacturer. E.g., Lenovo Type of the system. E.g., x86 or x64 Authority\Principle of the process associated with the flow. E.g., ACME\JSmith, <machine>\JSmith Authority\Principle of the parent of the process associated with the flow. E.g., ACME\JSmith, <machine>\JSmith Name of the process associated with the flow. E.g., firefox.exe SHA256 hash of the process image on disk associated with the flow. Name of the parent of process associated with the flow. E.g., explorer.exe SHA256 hash of the process image on disk of the parent process associated with the flow. Template Member E = Endpoint,
I = Interface,
F = Per Flow
(M) = Mandatory E, I, F (M) E E E E E F F F F F F 45112 / 12344 nvzFlowDNSSuffix string default 45113 / 12345 nvzFlowDestinationHostname string default 45114 / 12346 nvzFlowL4ByteCountIn unsigned64 totalCounter 45115 / 12347 nvzFlowL4ByteCountOut unsigned64 totalCounter 45116 / 12348 nvzFlowString string 45117 / 12349 45118 / 12350 nvzFlowFloat32 nvzFlowOctetArray float32 octetArray default default 45119 / 12351 nvzFlowOSEdition string default 45120 / 12352 nvzFlowModuleNameList basicList of nvzFlowString default 45121 / 12353 nvzFlowModuleHashList basicList of nvzFlowOctetArray default 45122 / 12354 nvzFlowCoordinatesList basicList of nvzFlowFloat32 default 45123 / 12355 nvzFlowInterfaceInfoUID unsigned32 identifier 45124 / 12356 nvzFlowInterfaceIndex unsigned32 default 45125 / 12357 nvzFlowInterfaceType unsigned8 default 45126 / 12358 nvzFlowInterfaceName string default 45127 / 12359 nvzFlowInterfaceDetailsList basicList of nvzFlowString default v2 fields default Per-­‐interface DNS suffix configured on the adapter associated with the flow for the endpoint. E.g., cisco.com The FQDN (hostname) that resolved to the destination IP on the endpoint. E.g., www.google.com Total number of incoming bytes on the flow at Layer4 (Transport). [payload only, without L4 headers] Total number of outgoing bytes on the flow at Layer4 (Transport). [payload only, without L4 headers] F Generic UTF-­‐8 string info-­‐element for lists Generic float32 info-­‐element for lists Generic octetArray info-­‐element for lists The OS Edition, such as Windows 8.1 Enterprise Edition List of 0 or more names of the modules hosted by the process that generated the flow. This can include the main DLLs in common containers such as dllhost, svchost, rundll32, etc. It can also contain other hosted components such as the name of the jar file in a JVM. List of 0 or more SHA256 hashes of the modules associated with the nvzFlowModuleNameList List of 32bit floating point values representing Accuracy, Latitude, Longitude, [Altitude] respectively. Altitude is optional. Coordinate based location information such as GPS, Wi-­‐
Fi Approximation, etc., Accuracy in meters – defines the error margin. Unique ID for an interface meta-­‐data. Should be used to lookup the interface meta-­‐data from the InterfaceInfo records. The index of the Network interface as reported by the OS. Interface Type, such as Wired, Wireless, Cellular, VPN, Tunneled, Bluetooth, etc. Enumeration of network types, defined by this spec. Network Interface/Adapter name as reported by the OS. List of name value pair (delimited by ‘=') of other interface attributes of interest. E.g., SSID=internet. (sent as part of a list) F F F (sent as part of a list) (sent as part of a list) E F F F F, I (M) I
I
I
I
Endpoint Exporter IPFIX / nvzFlow Templates An endpoint client exporter, such as Cisco AnyConnect NVM, sends IPFIX / Network Visibility Flow (nvzFlow) records based on the following templates. The template section lists the Information Elements in each template. All fields can be anonymized (“-­‐“) or completely absent unless indicated as MANDATORY. There are two revisions of the protocol. Version 2 (additional) elements are marked v2. Endpoint Identity Template (ID 262) The Endpoint Identity Template shall have the following Information Elements. virtualStationName (IPFIX standard information element : 350) nvzFlowUDID (MANDATORY) nvzFlowOSName nvzFlowOSVersion nvzFlowSystemManufacturer nvzFlowSystemType nvzFlowOSEdition (v2) Interface Info Template (ID 265) Interface Info template records identify each interface instance on the endpoint. Any change in to interface attributes should trigger a new Interface Info record to be sent with corresponding details. Each record (not just an interface) is distinctively identified by a unique ID – nvzFlowInterfaceInfoUID. Each Interface Info record shall have the following information elements: nvzFlowUDID (MANDATORY) nvzFlowInterfaceInfoUID (MANDATORY – v2) nvzFlowInterfaceIndex (v2) nvzFlowInterfaceType (v2) nvzFlowInterfaceName (v2) nvzFlowInterfaceDetailsList (v2) The exporter is expected to send the interface info record with a new UID if any of the attributes changes for an interface or for a new interface. E.g., if the same Wi-­‐Fi interface connects to a new SSID. The UID is expected to be unique on a given endpoint. InterfaceType has the following values: 1 – Wired Ethernet, 2 – Wireless (802.11), 3 – Bluetooth, 4 – Token Ring, 5 – ATM (Slip), 6 – PPP, 7 – Tunnel (generic), 8 – VPN, 9 – Loopback, 10 – NFC, 11 – Cellular, 15 – Unknown/Unspecified IPv4 Per-­‐Flow Data Template (ID 263) The Per-­‐Flow Data Template (IPv4) shall have the following Information Elements: protocolIdentifier (IPFIX standard information element : 4) (MANDATORY) sourceIPv4Address (IPFIX standard information element : 8) (MANDATORY) sourceTransportPort (IPFIX standard information element : 7) (MANDATORY) destinationIPv4Address (IPFIX standard information element : 12) (MANDATORY) destinationTransportPort (IPFIX standard information element : 11) (MANDATORY) flowStartSeconds (IPFIX standard information element : 150) (MANDATORY) flowEndSeconds (IPFIX standard information element : 151) (MANDATORY) nvzFlowUDID (MANDATORY) nvzFlowLoggedInUser nvzFlowProcessAccount nvzFlowProcessName nvzFlowProcessHash nvzFlowParentProcessAccount nvzFlowParentProcessName nvzFlowParentProcessHash nvzFlowL4ByteCountIn nvzFlowL4ByteCountOut nvzFlowDNSSuffix nvzFlowDestinationHostname nvzFlowInterfaceInfoUID (MANDATORY – v2) nvzFlowModuleNameList (v2) nvzFlowModuleHashList (v2) nvzFlowCoordinatesList (v2) IPv6 Per-­‐Flow Data Template (ID 264) The Per-­‐Flow Data Template (IPv6) shall have the following Information Elements. protocolIdentifier (IPFIX standard information element : 4) (MANDATORY) sourceIPv6Address (IPFIX standard information element : 27) (MANDATORY) sourceTransportPort (IPFIX standard information element : 7) (MANDATORY) destinationIPv6Address (IPFIX standard information element : 28) (MANDATORY) destinationTransportPort (IPFIX standard information element : 11) (MANDATORY) flowStartSeconds (IPFIX standard information element : 150) (MANDATORY) flowEndSeconds (IPFIX standard information element : 151) (MANDATORY) nvzFlowUDID nvzFlowLoggedInUser nvzFlowProcessAccount nvzFlowProcessName nvzFlowProcessHash nvzFlowParentProcessAccount nvzFlowParentProcessName nvzFlowParentProcessHash nvzFlowL4ByteCountIn nvzFlowL4ByteCountOut nvzFlowDNSSuffix nvzFlowDestinationHostname nvzFlowInterfaceInfoUID (MANDATORY – v2) nvzFlowModuleNameList (v2) nvzFlowModuleHashList (v2) nvzFlowCoordinatesList (v2) Data Model and Correlation IDs The following graphic shows the data model of the nvzFlow protocol and how the different correlation IDs are used to associate the different data collections. A collector will likely store 3 sets of data records, one for each collection of data. Analytics and reporting will correlate the 3 sets of data using the UDID as the key across the data sets. Additionally, for interface information, a correlation from the flow record to the interface information record will be done using the Interface UID. Implementation Considerations There are a number of considerations that should be evaluated as part of a solution implementation. Of particular note is data retention and data suppression by an exporter. For example, an exporter should have some means of securely preserving data when it is unable to communicate with the collector (such as when an endpoint is not on the network where the collector is located). Additionally, configurable protocol suppression should be considered for the endpoint exporter. Of particular note is broadcast and multicast traffic as this can be significant. An implementation should allow for the ability to also limit the export of any field that is not marked as mandatory if an administrator so chooses. Anonymization can be done at either the exporter or collector. Note that exporter anonymization would mean that the data will never be recoverable, whereas collector anonymization can preserve the data for forensic needs while still meeting privacy requirements. Exporter anonymization should be done by either excluding the field or sending an empty field with no data or alternatively using an indicative data element, such as (“-­‐“) for a string, to convey it was anonymized. Traffic that is bound to localhost should not be of consideration, unless some local network proxy function, on that endpoint, results in aggregation of flows. For example, a web security proxy running on the endpoint that binds web traffic flows to localhost for redirection to a cloud proxy, might be of interest for a local exporter implementation to consider. Performance impact should be a top consideration. In particular, when collecting and storing data locally or when sending bulk records that were previously stored on an endpoint. An exporter should act as a background service with low CPU and memory utilization. A collector should leverage lightweight hooking technologies and avoid approaches such as packet capture facilities so as to minimize system impact and reduce incompatibilities. Additionally, the exporter should throttle sending of cached records so as to not impact the normal network operation of a device. Caching limits should also be used to minimize impact on an endpoint device in terms of both storage and battery consumption. Future Considerations There are a number of future considerations for the nvzFlow protocol. The following are some possible items that will be considered in the near future: Compressed IPFIX records – Because the nvzFlow data set could exceed a single UDP packet, a future implementation might specify a template containing a single element that encapsulates a normal nvzFlow flow record in a compressed payload. A collector would first identify the record as containing the single type nvzFlowCompressedRecord and then expand that single payload into one of the nvzFlow template payloads described above. IPFIX over DTLS – Endpoints may be in places in the network where integrity, authenticity and privacy are a concern. As such, it is recommended that an exporter and collector support sending and receiving nvzFlow IPFIX records over DTLS (with the ability to fallback to TLS when DTLS is not possible due to network, firewall or proxy restrictions). Additional Exporter Formats – In addition to IPFIX, additional formats may be added, such as binary-­‐
JSON over HTTPs, to allow for a broader ecosystem of collector technologies. References 1. IP Flow Information Export (IPFIX) Entities [http://www.iana.org/assignments/ipfix/ipfix.xhtml] 2. Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information [http://tools.ietf.org/html/rfc7011] 3. Information Model for IP Flow Information Export (IPFIX) [http://tools.ietf.org/html/rfc7012] 4. Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements [http://tools.ietf.org/html/rfc5610] -­‐-­‐-­‐ End of document -­‐-­‐-­‐