Download CMon Installation Guide

Document Title: CMon Installation Guide
Document Title: CMon Installation Guide
Document Details
Activity:
SA4
Work Item:
Task1
Nature of
Deliverable:
O
Authors:
Trupti Kulkarni, Tobias Appel, Hao Yu
Dissemination:
PP (Project Participants)
Document Title: CMon Installation Guide
Table of Contents
1
Introduction...................................................................................................................................2
1.1 Prerequisites table ................................................................................................................3
2
CMon System Installation ............................................................................................................4
2.1 CMon server specifications ..................................................................................................4
2.1.1 Hardware .................................................................................................................4
2.1.2 Ports 4
2.2 Installation of prerequisites ...................................................................................................4
2.2.1 Python installation ...................................................................................................5
2.2.2 DB installation..........................................................................................................5
2.3 CMon installation ..................................................................................................................5
2.3.1 CMon HQ (Central Instance) ...................................................................................5
2.3.2 CMon GUI (Central Instance) ..................................................................................6
2.3.3 CMon AGT (Distributed) ..........................................................................................6
2.4 Upgrading CMon ..................................................................................................................6
2.5 CMon configuration ..............................................................................................................6
2.5.1 CMon HQ .................................................................................................................7
config.properties
7
2.6
2.7
[Database Information]
7
logger.conf
7
2.5.2 CMon GUI................................................................................................................7
2.5.3 CMon AGT ...............................................................................................................8
logger.conf
9
Starting CMon .......................................................................................................................9
2.6.1 CMon HQ .............................................................................................................. 10
2.6.2 CMon Agent.......................................................................................................... 10
Shutting down the service ................................................................................................. 10
2.7.1 CMon HQ .............................................................................................................. 10
2.7.2 CMon Agent.......................................................................................................... 10
3
Appendix .................................................................................................................................... 11
3.1 Net SNMP .......................................................................................................................... 11
3.1.1 SNMP Polling ....................................................................................................... 11
3.1.2 SNMP Traps ......................................................................................................... 11
3.2 CMon GUI Apache Configuration (/etc/httpd/conf.d/cmon.conf) ....................................... 12
3.3 Mongo DB Configuration ................................................................................................... 13
4
Glossary ..................................................................................................................................... 14
5
References ................................................................................................................................. 15
Page 1
Document Title: CMon Installation Guide
Introduction
1
This document describes the prerequisites for CMon (Multi-Domain Circuit Monitoring System), and specifies
instruction to install and configure CMon. The system has both central and distributed components; hence the
instructions below cater to each component separately.
CMon is a distributed multi-domain circuit monitoring system. It fetches monitoring data from domains and
splices different segment performances to show for the end-to-end link. CMon is able to provide end-to-end
circuit monitoring services with flexibility, extensibility, and vendor independency, regardless of the underlying
circuit provisioning systems (CPSs).
CMon System:
Distributed components:
-
CMon Agents (AGT) collect monitoring information, in order to collate for a higher, end-to-end service.
This system should be deployed in each involved domain.
Central components: These are installed only once centrally.
- CMon Headquarters (HQ) – Required to collect information from different AGTs and aggregate it in order
to show for the complete circuit.
-
CMon Graphical User Interface (GUI) – Required to display the monitoring information thus collected by
AGTs and collated by HQ.
The diagram below shows the communication between different components of CMon.
Page 2
Document Title: CMon Installation Guide
1.1
Prerequisites table
Below is a table with an overview of prerequisites for installing each part of the software (details are given in the
corresponding sections):
Prerequisite
CMon HQ (single, central
instance)
CMon AGT
(distributed
instances in each
domain)
CMon GUI (single,
central instance)
Python 2.6 or 2.7 (3.x
unsupported)
Required
Required
Required
Apache
Not required
Not required
Required
Mongo DB 2.6
Required
Not required
Not required
(see section 2.2.2)
Supporting Python libraries
cStringIO
logging
logging
netsnmp-python
pycurl
pyserial
pyserial
xml
xml
xmllib
xmllib
xmlrpclib
xmlrpclib
zdaemon
zdaemon
NTP Time Synchronisation
Required
Required
Required
Net-SNMP
Required, if using SNMP
option
Not required
Not required
Page 3
Document Title: CMon Installation Guide
2
CMon System Installation
2.1
CMon server specifications
The servers that host CMon Graphical User Interface (GUI), CMon Headquarters (HQ) and the CMon Agent
(AGT) must fulfil the following hardware requirements
2.1.1





2.1.2







2.2
Hardware
Central Processing Unit (CPU) – minimum 4 gigahertz (GHz).
RAM – minimum 4 gigabyte (GB).
Disk space – minimum 1 GB, recommended 5 GB (for long term logs and if using SNMP traps).
Network Interface Controller (NIC) – 1 Fast Ethernet NIC.
Operating System (OS) – The software has been tested on Red Hat and Debian Linux, which are the
recommended ones.
Ports
8080 – Required for CMon HQ only, this is the port on which the HQ system’s SOAP server listens on,
for incoming CRM and CTM messages from automatic provisioning systems.
443 – Secure port for communication between central HQ and distributed AGTs.
27017 – MongoDB database access, and port for communication between GUI and server hosting the
database (typically the HQ).
161 – Required on AGT only if using SNMP polling option.
162 – Required on AGT only if using SNMP traps option.
25 – Simple Mail Transfer Protocol (SMTP) for sending mails with notifications (optional).
80 – Required on GUI server for HTTP
Installation of prerequisites
Please consult the table in section 1.1 in order to identify the appropriate prerequisites.
Important: NTP based time synchronization is important across all domains using CMon system. Multi-domain
provisioning systems send circuit reservation notifications based on the time specified, and for CMon to start
monitoring at the time, it is important that hosts that deploy and use any part of CMon are synchronised.
Page 4
Document Title: CMon Installation Guide
2.2.1
Python installation
Download and install Python from the website: https://www.python.org/downloads/
You may need to employ other methods to install Python, based on the OS distribution flavour on the servers,
such as ’apt-get’ for Debian, or ’yum install’ for Red Hat etc. Please note that only Python 2.6 and 2.7
are supported at this time.
2.2.2
MongoDB installation
Install Mongo DB 2.6 or higher from https://www.mongodb.org/downloads and follow the installation instructions
as per the OS distribution.
The current CMon system only works with MongoDB, and hence no other RDBMS or No-SQL databases can
be used. The choice of No-SQL was due to the ease of being able to store any parameters of monitoring data
as well as supporting different circuit types, without the restriction of schema-based RDBMSs.
If you installed MongoDB using a tarball it is recommended to start it using this command:
mongod --config /etc/mongod.conf
For recommended default options for mongod.conf, please check the Appendix at the end of this document.
There is no need to create a database explicitly in MongoDB. The CMon system will read the database name
from the HQ’s configuration file (see section 2.5.1), and create the database on its first execution.
As it is a No-SQL database, there is no schema file definition either.
2.3
CMon installation
2.3.1
CMon HQ (Central Instance)
This assumes you have downloaded the latest cmonhq.tar.gz which contains the desired CMon HQ release.
CMon HQ should be installed at a single site hosting the CMon system.

In a location of your choice, please extract cmonhq.tar.gz as follows to install CMon HQ. The
distribution contains everything in a single folder named cmonhq
tar xvfz cmonhq.tar.gz


Put the cmonhq folder in a location of your choice (e.g. /opt/cmonhq) but do not rename it
Ensure that the configuration files are updated as per the deployment environment. Configuration files
are stored under cmonhq/conf
 Please see the Appendix for explanations of the various configuration options
 After you have adapted the configuration files to your needs, you can start CMon HQ using this
command:
sudo ./cmonhqserver.py start
Page 5
Document Title: CMon Installation Guide
2.3.2
CMon GUI (Central Instance)
This assumes you have downloaded the cmonfront.zip which contains the CMon GUI in Web2py web
framework.
 Unzip the file in /var/www/cmonfront
 Go to /var/www/cmonfront/applications/cmon/static/conf/ to configure
preferences.conf
 Configure your Apache configuration file for CMon GUI: (see more in Section 3.2 or
http://web2py.com/books/default/chapter/29/13/deployment-recipes#Linux-and-Unix)
 Alternative Web servers like nginx might work, but are unsupported
2.3.3
CMon AGT (Distributed)
This assumes you have downloaded the cmonagent.tar.gz which contains the desired CMon AGT release. This
should be installed in all participating domains that intend to use CMon for multi-domain circuit monitoring.

In a location of your choice, please extract cmonagent.tar.gz as follows to install the AGT. The
distribution contains everything in a single folder named cmonagent.
tar xvfz cmonagent.tar.gz




2.4
Put the cmonagent folder to a location of your choice (e.g. /opt/cmonagent) but do not rename it
Ensure that the configuration files are updated as per the deployment environment. All configuration
files are stored under cmonagent/conf
Please see the Appendix for explanations on the various configuration options
After you have modified the configuration to suit your needs you can start the agent by running the
following command:
sudo ./cmonagentserver.py start
Upgrading CMon
The process for upgrading any CMon component is the same as installing it, with the exception of reconfiguring the configuration files. Backup the configuration files and simply un-compress the component in a
folder of choice. Stop CMon HQ / Agent service by typing ./cmonhqserver.py stop
or ./cmonagentserver.py stop. Now you can replace the old installation with the updated files. When
done, restore the backed-up configuration files in their relevant locations. Start the CMon system as specified in
section 2.3.
Please note, that in case of GUI, Apache (httpd) should be restarted before the new GUI is ready to be used.
2.5
CMon configuration
CMon configuration files can be found in the conf directory in the distribution for HQ and AGT, and in [TODO]
directory for the GUI. It is a well-known practice to separate a config file in Python application into sections, so
Page 6
Document Title: CMon Installation Guide
as to provide a clear demarcation based on usage. Following listing describes the properties that should be set
to run the application properly, along with the sections they appear in.
2.5.1
CMon HQ
config.properties
[Database Information]
 db_hostname – hostname or IP address of where the MongoDB service is running, typically the
CMon HQ server
 db_name – name of database; as MongoDB is a schema-less DB, database name can be created
on the fly. Once created though, it cannot be changed without losing information present in earlier
database
[Query Scheduler]
 config_update_interval – how often should the HQ check for updated AGT information, in
seconds
[Networking]
 socket_timeout – timeout, in seconds, if no response received from AGT
[SOAP Server]
 soap_address – bind IP address for the SOAP server to listen on
 soap_port – port on which the SOAP messages will be sent and received
[Log Configuration File]
 log_conf – absolute path of HQ log configuration file
[Trapper]
 trapper_address – bind IP address on which HQ will listen for incoming connections from AGT
 trapper_port – port on which HQ listens to processed SNMP traps sent by AGT
logger.conf
 Retain this file as given in the distribution, with the exception of updating location of log file, if
required.
2.5.2
CMon GUI
preferences.conf
[DateTimeFormat]
 Format – time format as seen on the GUI, typically %d-%b-%Y %H:%M
[DBAddress]
 Address – DB URL for web2py framework, typically sqlite://storage.sqlite
Page 7
Document Title: CMon Installation Guide
[MongoDBAddress]
 Address – hostname or IP address where Mongo DB is installed, typically that of HQ
o When IP is used, you can write, e.g. 10.10.10.10
o When hostname is used,you should follow the mongodb URI format. The following is the
standard URI connection scheme:
mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
The components of this string are:
 mongodb://
A required prefix to identify that this is a string in the standard connection format.
 username:password@
Optional. If specified, the client will attempt to log in to the specific database
using these credentials after connecting to the mongod instance.
 host1
This the only required part of the URI. It identifies a server address to connect to.
It identifies either a hostname, IP address, or UNIX domain socket.
 :port1
Optional. The default value is :27017 if not specified.
 hostX
Optional. You can specify as many hosts as necessary. You would specify
multiple hosts, for example, for connections to replica sets.
 :portX
Optional. The default value is :27017 if not specified.
 /database
Optional. The name of the database to authenticate if the connection string
includes authentication credentials in the form of username:password@. If
/database is not specified and the connection string includes credentials, the
driver will authenticate to the admin database.
 ?options
Connection specific options. See Connection String Options for a full description
of these options.
If the connection string does not specify a database/ you must specify a slash
(i.e. /) between the last hostN and the question mark that begins the string of
options.
More can be found here (http://docs.mongodb.org/manual/reference/connection-string/)
 Port – port on which the GUI communicates with MongoDB on HQ, typically 27017
[RefreshInterval]
 Value – value, in seconds, after which GUI refreshes
[SearchWindowSize]
 Size – maximum number of data records to display.
[StepWidthMB]
 Width – step value for y-axis when utilization is selected.
2.5.3
CMon AGT
agent_config.conf
Page 8
Document Title: CMon Installation Guide
[Operation]
 MODE – can be set to POLL (for a polling AGT) or TRAP (for a trap processing AGT)
[Domain Name]
 Domain_Name – name of domain where AGT is installed; for example; geant.net for AGT
installed in GEANT
[ServerInfo]
 HOST_IP – bind IP on which AGT listens for incoming connections from HQ
 HOST_PORT – port on which AGT communicates with HQ, typically 443
[Log Configuration File]
 Log_Conf – absolute path of AGT log configuration file
[DevicesPropertiesInfo]
 File_Name – absolute path of devices.properties file
[SNMP Poll Options]
 SNMP_Community_String – community string for SNMP polling requests
 Device_Type – whether layer 2 switch (L2Switch), or layer 3 router (L3Router)
[SNMP Trap Options]
 Proxy_Address – IP address or hostname to where SNMP traps are processed and sent to,
typically the HQ. Also takes the port value, typically 443, immediately after this, separated by a
colon (:)
 SNMP_Traps_Path - Absolute path where SNMP traps will be written to on receipt
 SNMP_Traps_Archive_Path - Absolute path where SNMP traps will be archived
 SNMP_Traps_Unwanted_Path - Absolute path where unwanted SNMP traps will be stored
[DataInfo]
 File_Name – name of XML file with monitoring data
 Monitoring_Data_Path – absolute path where above file is located
logger.conf
 Retain this file as given in the distribution, with the exception of updating location of log file, if
required.
2.6
Starting CMon
Both the central component HQ and distributed component AGT need Python in the path to start the service.
This should already be installed as per section 2.2.1
Services are started on port 443.
Page 9
Document Title: CMon Installation Guide
2.6.1
CMon HQ
In the distribution directory type:
sudo python cmonhqserver.py start
2.6.2
CMon Agent
In the distribution directory type:
sudo python cmonagentserver.py start
sudo python cmonagent-trapper.py start
2.7
Shutting down the service
2.7.1
CMon HQ
In the distribution directory type:
sudo python cmonhqserver.py stop
2.7.2
CMon Agent
In the distribution directory type:
sudo python cmonagentserver.py stop
sudo python cmonagent-trapper.py stop
Page 10
Document Title: CMon Installation Guide
3
Appendix
3.1
Net SNMP
If a domain chooses to use either SNMP polling or traps as an option to allow the local CMon AGT to collect
monitoring data, then Net SNMP must be installed as a pre-requisite as mentioned in section 1.1. (link to
download: http://www.net-snmp.org/download.html
3.1.1
SNMP Polling
To allow CMon AGT to poll devices in the domain as listed in devices.properties, appropriate MIBs
should be installed in default directory as mandated by Net SNMP installation. The following file should be
updated as a result:
snmp.conf – absolute path to the directory where MIB files are present.
3.1.2
SNMP Traps
If a domain using CMon AGT chooses to provide monitoring data to it via SNMP traps, then following files must
be updated:
snmptrapd.conf – This file should have the following contents:
disableAuthorization yes
traphandle default $CMONAGENT/simple_trap_handler.sh <absolute path of SNMP traps>
where:
$CMONAGENT – the installation directory of CMon AGT
<absolute path of SNMP traps> - where traps will be logged first on receipt, as configured in
SNMP_Traps_Path in agent_config.conf, detailed in section 2.5.3
Then, snmptrapd should be started as a service, which listens on port 162 for incoming SNMP traps.
Page 11
Document Title: CMon Installation Guide
3.2
CMon GUI Apache Configuration
3.2.1
/etc/httpd/conf.d/cmon.conf
To set up web2py with mod_wsgi, create a new Apache configuration file and include the following code for
cmonfront: (more instructions about web2py deployment can be found at:
http://web2py.com/books/default/chapter/29/13/deployment-recipes#Linux-and-Unix
NameVirtualHost *:80
NameVirtualHost *:443
<VirtualHost *:80>
ServerName <server that hosts CMon GUI>
Redirect / <https://server that hosts CMon GUI>
</VirtualHost >
<VirtualHost *:443>
ServerName <server that hosts CMon GUI>
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/cmon-gui.crt
SSLCertificateKeyFile /etc/pki/tls/private/cmon-gui.key
SSLProtocol all –SSLv2 –SSLv3
WSGIDaemonProcess web2py user=apache group=apache processes=1 threads=1
WSGIProcessGroup web2py
WSGIScriptAlias / /var/www/cmonfront/wsgihandler.py
<Directory /var/www/cmonfront>
AllowOverride None
Order Allow,Deny
Deny from all
<Files wsgihandler.py>
Allow from all
</Files>
</Directory>
AliasMatch ^/([^/]+)/static/(?:_[\d]+.[\d]+.[\d]+/)?(.*) /var/www/cmonfront/applications/$1/static/$2
<Directory /var/www/cmonfront/applications/*/static>
Options -Indexes
ExpiresActive On
ExpiresDefault "access plus 1 hour"
Order Allow,Deny
Allow from all
Page 12
Document Title: CMon Installation Guide
</Directory>
CustomLog /var/log/httpd/access_log common
ErrorLog /var/log/httpd/error_log
</VirtualHost>
3.2.2
/etc/httpd/conf.d/wsgi.conf
WSGISocketPrefix run/wsgi
3.2.3
Directory permission
chown –R apache:apache /var/www/cmonfront/
To make sure all files in the cmonfront directory have the apache:apache (www-data:www-data for Ubuntu)
3.3
Mongo DB Configuration
The configuration file to start mongod as service should have the following values at minimum:
# Where to log
logpath=/var/log/mongod.log
logappend=true
# Fork and run in background
fork=true
dbpath=/var/lib/mongo/DB/LinkDB
# Location of pidfile
pidfilepath=/var/run/mongodb/mongod.pid
# Interface to listen on. Comment out to listen on all interfaces.
bind_ip=0.0.0.0
# Disable/enable scripting engine
noscripting=true
# Use smaller default data file size; else dbpath should have at least 3.5G of
free space
smallfiles=true
# Port to use
port=27017
Page 13
Document Title: CMon Installation Guide
4
Glossary
CMon
GÉANT Multi-Domain Circuit Monitoring
CMon AGT
CMon Agent
CMon GUI
CMon Graphical User Interface
CMon HQ
CMon Headquarters
RDBMS
Relational DataBase Management System
SNMP
Simple Network Management Protocol
SQL
Structured Query Language
Page 14
Document Title: CMon Installation Guide
5
References
[CMon]
CMon, http://forge.geant.net/forge/display/cmon/
[MongoDB]
MongoDB, https://www.mongodb.org/
[Net-SNMP]
Net-SNMP, http://www.net-snmp.org/
[XML RPC]
Remote Procedure Call protocol that uses eXtensible Markup Language for encoding,
http://tldp.org/HOWTO/XML-RPC-HOWTO/xmlrpc-howto-intro.html
Page 15