Download TECHNOLOGY-DRIVEN METRICS

Metrics
SABSA High-Level Framework
Gap Analysis
The difference between
where you are and
where you want to be:
 # malware
infections/month
 Rate of finding illegal
software, hardware
 Security awareness
training averages
SEI/COBIT Level 4 Monitoring:
Includes Metrics
Metrics inform management (and
independent auditors) of the effectiveness
of the security program
 Monitoring achievement of control
objective may be more important than
perfecting security procedures

Which metrics to use?
Business-Driven
Addresses specific business
risks
 Inherent industry risks
 Tailored to organization
 Measures adherence to
control objectives
Technology-Driven
Addresses recent threats
observed by CERT
 CERT: Computer
Emergency Readiness
Team
 Addresses recent
forensic data
Monitoring Function:
Business-Driven Metrics
Executive mgmt is interested in
Strategic risk, budget, policy.
Metrics Review every 6 months-1 year
Metrics
Tactical
Metrics
Determine effectiveness of
security program: risk changes,
compliance, incident response tests.
Review quarterly to half-year
Operational
Metrics Technical details:
E.g., firewall, logs, IPS,
vulnerability tests.
Review weekly.
Automate statistics.
Monitoring Function:
Business-Driven Metrics
Project Plan or Budget Metrics
Strategic Risk performance
Metrics Disaster Recovery Test results
Audit results
Regulatory compliance results
Metrics
Tactical
Metrics
Policy compliance metrics
Exceptions to policy/standards
Changes in process or system
affecting risk
Incident management effectiveness
Operational
Metrics Vulnerability Scan results
Server config. standards
compliance
IDS monitoring results
Firewall log analysis
Patch mgmt status
Which metrics?
Step 1: What are the most important security
areas … threats …. regulation … to monitor in
your organization?

Step 2: Which metrics make the most sense
to collect. Can they be automated?


Step 3: Consider the 3 perspectives: strategic,
tactical, operational metrics, relative to 3
audiences.
Monitoring Function: Metrics
Risk:
The aggregate ALE
% of risk eliminated, mitigated,
transferred
# of open risks due to inaction
Cost Effectiveness:
What is:
Cost of workstation security per user
Cost of email spam and virus
protection per mailbox
Operational Performance
Time to detect and contain incidents
% packages installed without problem
% of systems audited in last quarter
Organizational Awareness:
% of employees passing quiz, after
training vs. 3 months later
% of employees taking training
Technical Security Architecture
# of malware identified and neutralized
Types of compromises, by severity &
attack type
Attack attempts repelled by control
devices
Volume of messages, KB processed
by communications control devices
Security Process Monitoring:
Last date and type of BCP, DRP, IRP
testing
Last date asset inventories were
reviewed & updated
Frequency of executive mgmt review
activities compared to planned
Workbook: Metrics
Metrics Selected
What are the most important areas to monitor in your organization?
Lunatic gunman
FERPA Violation
Category
Major Risks:
Metric
Operational
Web Availability
Calculation & Collection
Method
Period of
Reporting
Information Tech. Group
1 year
Cost of incidents
Incident Response totals
6 months
% employees passing FERPA
quiz
Annual email requesting
testing
1 year
% employees completing
FERPA training
Two annual trainings with
1 year
sign-in. Performance review
# Hours Web unavailable
Incident Response form
6 months
# brute force attacks
Incident Response form
1 month
# malware infections
Incident Response form
1 month
Strategic Cost of security/terminal
Tactical
Cracking Attempt
CERT-Recommended
TECHNOLOGY-DRIVEN
METRICS
SANS: Critical Controls for
Effective Cyber Defense
Metric: Temporarily install unauthorized
software/hardware on a device. It should
be:
 found within 24 hours (or 2 minutes?)
 isolated within one hour confirmed by
alert/email
 reported every 24 hours until device is
removed.
SANS Critical Control 1:
Inventory of Authorized Devices
Ensure all devices (with IP address) on network
are known, configured properly, and patched.
 Scan network daily or use DHCP reports or
passive monitoring.
 Compare results with baseline configuration.
 Metric: Temporarily install unauthorized device.
SANS: Critical Control 2:
Inventory of Authorized Software
Ensure all software is approved and recently patched
 Whitelist defines the permitted list of software.
 Blacklist defines illegal software (e.g., IT tools).
 Endpoint Security Suites (ESS) contain antivirus,
antispyware, firewall, IDS/IPS, s/w white/blacklisting.
 Metric: Temporarily install unauthorized software
on a device.
SANS Critical Control 3:
Secure Configurations for Hardware &
Software






All devices are hardened using recommended security
configurations
Illegal software list exists, includes Telnet, VNC, RDP
New software is quarantined and monitored.
Imaged software is maintained in an updated state.
Build secure images, and use configuration checking
tools daily.
Metric: Temporarily attempt to change a set of random
configurations.
SANS Critical Control 4:
Continuous Vulnerability Assessment
Run vulnerability scans on all systems at least weekly, preferably
daily. Problem fixes are verified through additional scans.
 Vulnerability scanning tools (updated) for: wireless, server,
endpoint, etc.
 Automated patch management tools notify via email when all
systems have been patched.

Metric: If the scan does not complete in 24 hours, an email
notification occurs.
SANS Critical Control 5:
Malware Defense
Antivirus/antispyware is always updated
 Run against all data


Additional controls: blocking social media, limiting
external devices (USB), using web proxy gateways,
network monitoring.


shared files, server data, mobile data.
Endpoint security suites can report tool is updated and active on
all systems
Metric: For benign malware (e.g., security/hacking tool)
install, antivirus prevents installation or execution or
quarantines software

sends an alert/email within one hour indicating specific device
and owner
SANS Critical Control 6:
Application S/W Security
New application software is tested for security vulnerabilities:
 Web vulnerabilities: buffer overflow, SQL injection, cross-site
scripting, cross-site request forgery, clickjacking of code, and
performance during DDOS attacks.
 S/W validates input for size, type
 S/W does not report system error messages directly
 Automated testing includes static code analyzers and automated
web scanning.
 Configurations include application firewalls and hardened
databases.
 Metric: An attack on the software generates a log/email within 24
hours (or less).
 Automated web scanning occurs weekly or daily
SANS Critical Control 7:
Wireless Device Control
Wireless access points are securely configured with WPA2 protocol and AES
encryption.
 Extensible Authentication Protocol-Transport Layer Security (EAP/TLS)
provides mutual authentication.
 Only registered, security-approved devices are able to connect
 Wireless networks are configured for the minimum required radio
footprint.
 Metrics: Wireless intrusion detection systems detect available wireless
access points and deactivate rogue access points within 1 hour
 Vulnerability scanners can detect unauthorized wireless access points
connected to the Internet.
SANS Critical Control 8:
Data Recovery Capability
Backups are maintained at least weekly and
more often for critical data.
 Backups are encrypted and securely stored.
 Multiple staff can perform backup/recovery.
 Metric: Test backups quarterly for a random
sample of systems. This includes operating
system, software, and data restoration.
SANS Critical Control 9:
Security Skills Assessment
Security awareness training is required for end users,
system owners.
Security training is necessary for programmers, system,
security and network administrators.
 Metric: Test security awareness understanding

Periodically test social engineering tests via phishing
emails and phone call

Employees who fail a test must attend a class
SANS Critical Control 10:
Secure Network Configurations
A configuration db tracks approved configurations in
config. mgmt. for network devices: firewalls, routers,
switches.
 Two-factor identification is used for network devices.
 Tools: Tools perform rule set sanity checking for
Access Control Lists.
 Metric: Any change to the configuration of a
network device is reported within 24 hours
SANS Critical Controls
11. Control of Network
Ports, Protocols, and
Services:
 Default Deny packets.
 Periodically review for
restriction
12. Controlled
Administrative
Privilege:
 Minimal elevated
privileges.
 Passwords are
complex, changed
periodically, 2-factor
authentication.
SANS Critical Controls
13. Boundary
Defense:
 Use firewall zones to
filter incoming and
outgoing traffic.
 Blacklist & whitelist
network addresses
14. Analysis of
Security Audit Logs:
 Server logs are writeonly and archived for
months.
 Firewalls log all
allowed and blocked
traffic.
 Unauthorized access
attempts are logged.
SANS Critical Controls
15. Need to Know Access:
Prevent exfiltration of
data to competitors.
 Data classification
 Restrictive firewall
configurations
 Logged access to
confidential data
16. Account Monitoring and
Control:
Terminated accounts -> removed
Expired password/ disabled/
locked out accounts, ->
investigated
Failed logins -> lockouts;
Inactivity -> locked sessions
Unusual time access -> alert.
Data exfiltration recognized by
keywords.
SANS Critical Controls
17. Data Loss
Prevention:
 Prevent exfiltration of
proprietary or
confidential info.
 Encryption of mobile
and USB devices;
 Disable USB
18. Incident
Response:
 Incident Response
Plan defines who
does what for various
conditions.
 IRP includes contact
information for third
party contractors.
SANS Critical Controls
19. Secure Network
Engineering:




Separate zones: DMZ,
middleware, private network
DMZ accessed through proxy
firewall
DMZ DNS is in DMZ; internal
DNS is in internal zone, …
Emergency config. for
restricted network is ready for
quick deployment.
20. Penetration
Tests:


Penetration tests =
vulnerability tests +
attacker tests.
Red Team exercises
test incident
response team
reactions.
Question
The difference between where an
organization performs and where they
intend to perform is known as:
1. Gap analysis
2. Quality Control
3. Performance Measurement
4. Benchmarking
Question
The MOST important metrics when
measuring compliance include:
1. Metrics most easily automated
2. Metrics related to intrusion detection
3. Those recommended by best practices
4. Metrics measuring conformance to policy
Reference
Slide #
Slide Title
Source of Information
Level 4: Monitoring: Includes Metrics
CISM: page 192 -194
Monitoring Function: Metrics
CISM: page 192