Download Robert_Bolton_Nagios_2012

Custom SNMP OID Creation for System Monitoring
Robert V. Bolton
[email protected]
About Robert V. Bolton
Systems Administrator
Center for High Performance Computing @ University of Utah
Mac Desktops, Linux Servers, HP Networking
Infrastructure Monitoring: Nagios and Cacti
Pyton Coder
Nagios Plugins and Cacti Scrtips
Student
Computer Engineering or Electrical Engineering
Expected Graduation = ?
Geek
Board Game Enthusiast
Amateur Radio Operator (KE7ZEA)
www.robertvbolton.com
2012
2
What we’re going to cover today.
Why Bother Creating Custom OIDs
OID Tree Structure
Python Module: snmp_passpersist
Real World Example: IOStat Statistics
2012
3
Why Bother Creating Custom OIDs
• SNMP is “simple” to use to gather system
statistics.
• Nagios Plugins work great with Nagios, but may
not provide data to other monitoring software.
• Example: NRPE Remote Execution.
• Offload Time Consuming System Checks.
• Provides Statistics for things not normally
provided by in the SNMP tree.
2012
4
OID Tree Structure
• SNMP uses a hierarchical tree structure
2012
OID Tree Structure Conitued
• Numbers are used to locate information
• Each number corresponds to a specific branch
of the OID tree.
• Management Information Base (MIB) files map
OID numbers to human readable format
• .1.3.6.1.4 is the top of the private branch
• Vendor OIDs
• Our Custom OIDs
2012
Python Module: snmp_passpersist
• Why Python…Because I know Python!
• I believe it is possible to create OIDs with Perl as
well if your prefer.
• Github: nagius/snmp_passpersist
• Requires Net-SNMP
• Installation is easy: Download source, python
setup.py install, and you’re done.
• Import: import snmp_passpersist as snmp
2012
Python Module: snmp_passpersist contiued
#!/usr/bin/python -u
import snmp_passpersist as snmp
from commands import getoutput
base_oid=".1.3.6.1.4.1234.1.3"
def get_file_systems():
file_systems = getoutput("df -iP | awk '{print $1}'").split('\n')
file_systems.remove('Filesystem')
return file_systems
def get_inode_count():
inode_count = getoutput("df -iP | awk '{print $3}'").split('\n')
inode_count.remove('IUsed')
return inode_count
def update():
file_systems = get_file_systems()
inode_count = get_inode_count()
counter = 0
for x in file_systems:
counter += 1
pp.add_int("1.%s" % str(counter),counter)
pp.add_str("2.%s" % str(counter), x)
counter = 0
for x in inode_count:
counter += 1
pp.add_gau("3.%s" % str(counter), x)
pp=snmp.PassPersist(base_oid)
pp.start(update,60)
2012
Python Module: snmp_passpersist contiued
Add to /etc/snmp/snmpd.conf
pass_persist .1.3.6.1.4.1234.1.3 /usr/local/bin/inodeCount.py
Results
SNMPv2-SMI::private.1234.1.3.1.1 = INTEGER: 1
SNMPv2-SMI::private.1234.1.3.1.2 = INTEGER: 2
SNMPv2-SMI::private.1234.1.3.1.3 = INTEGER: 3
SNMPv2-SMI::private.1234.1.3.2.1 = STRING: "/dev/mapper/winkler-root"
SNMPv2-SMI::private.1234.1.3.2.2 = STRING: "/dev/sda1"
SNMPv2-SMI::private.1234.1.3.2.3 = STRING: "tmpfs"
SNMPv2-SMI::private.1234.1.3.3.1 = Gauge32: 355793
SNMPv2-SMI::private.1234.1.3.3.2 = Gauge32: 48
SNMPv2-SMI::private.1234.1.3.3.3 = Gauge32: 1
2012
Real World Example: Iostat
• Problem
• User: “The network is slow!”
• Support: To many users hammering on a file
system
• Solution Requirements
• Present Users with easy to read graphs of disk
statistics.
• Allow for Nagios monitoring of disk statistics
• Minimal Impact of monitoring because of Iostat
2012
Real World Example: Iostat
• Iostat
• First output from Iostat is garbage.
• Cron Job runs Iostat and stores the results in a
temp file
• Python Script iostat.py
• Reads results from temp file and uses
snmp_passpersist to update a custom branch of
OID tree.
• SNMP OID
• Data accessible to both Nagios and Cacti.
2012
Thank you!
www.chpc.utah.edu
Questions?
Robert V. Bolton
[email protected]