Download Is finding security holes a good idea?

Is finding security holes a
good idea?
Presented By: Jeff Wheeler
CSC 682
Outline
•
•
•
•
•
•
Introduction
Vulnerability Lifecycle
Cost of Disclosure
Finding rate to pr
Rate of Vulnerability Discovery
Sources of Error
Introduction
•
Assertions
1. It is better for vulnerabilities to be found by
good guys than bad guys.
2. Vulnerability finding increases total software
quality
The life cycle of a vulnerability
• Introduction – the vulnerability is first
released as part of the software.
• Discovery – the vulnerability is found.
• Private Exploitation – the vulnerability is
exploited by the discoverer or a small
group known to him or her.
• Disclosure – a description of the
vulnerability is published.
The life cycle of a vulnerability
• Public Exploitation – the vulnerability is
exploited by the general community of
black hats.
• Fix Release – a patch or upgrade is
released
The life cycle of a vulnerability
• These events do not occur strictly in this
order.
– Ex: software manufacture releases disclosure
and fix
White Hat Discovery
• Discovery, Fix, and Disclosure: Best Case
– The vulnerability is discovered by a
researcher with no interest in exploiting it.
– The researcher notifies the vendor
– The vendor releases an advisory and a fix
– Public exploitation begins at time of disclosure
White Hat Discovery
Black Hat Discovery
• Discovery, Fix, and Disclosure: Worst
Case
– The vulnerability is first discovered by
someone with an interest in exploiting it.
– Black hat community exploitation
– Knowledgeable person identifies exploit being
used against a system and notifies vendor
– The vendor releases an advisory and a fix
– Public exploitation begins at time of disclosure
Black Hat Discovery
WHD versus BHD
• WHD eliminates period of Private
Exploitation
• CBHD – CWHD = Cpriv
• Are administrators more likely to patch if
they know a vulnerability is being actively
exploited?
– Total number of vulnerable systems will
decline more quickly, minimizing peak
exploitation rate
Cost-Benefit Analysis of Disclosure
• Best Case
– White hat discovery, never rediscovered or
exploited
• Worst Case
– Black hat discovery
• Cpriv + Cpub
Cost-Benefit Analysis of Disclosure
From finding rate to pr
• Assumption: Vulnerability discovery is a
stochastic process.
– Overall rate of vulnerability discovery in a
particular application is a good estimate for pr
– Pr upper bound current percent discovery
Determining the Vulnerability
Discovery Rate
• Assumption: Software undergoes multiple
releases
– If we assume patches/releases do not
introduce new bugs, only fixes, we can
assume overall software quality increases
with time
• How does one determine this rate?
Determining the Vulnerability
Discovery Rate
• ICAT vulnerability metabase
– A searchable index of computer
vulnerabilities.
– Entire database available for public download
and analysis
• Relevant Information
– Rate of discovery over time, Program and
version effected
• Data Cleansing
Sources of Error
•
•
•
•
•
Unknown Versions
Bad Version Assignment
Announcement Lag
Severity of Vulnerabilities
Operating System Effects
– Packages included with OS, use OS release date
instead of package release date
• Effort Variability
• Different Vulnerability Classes
• Data Errors
Is it worth disclosing
vulnerabilities?
• If there is no depletion of vulnerabilities,
then disclosing vulnerabilities is always
harmful. This implies there is an infinite
number of vulnerabilities and pr
approaches zero.
• If we assume the pool of vulnerabilities is
depleting, and all vulnerabilities will
eventually be discovered, pr=1, and
disclosing vulnerabilities makes sense.
Conclusions
• This research does not provide sufficient
evidence that vulnerability finding and
disclosure provides in increase in software
security sufficient to offset the effort being
invested.
• This research does not provide sufficient
evidence that vulnerability finding and
disclosure is a bad idea.
Conclusions
• Prefer continuous white hat discovery with
no disclosure until exploitation by black
hat?
• How do we estimate the number of
vulnerabilities in an application, both
discovered and undiscovered?