Download JMX- Monitoring and Managing of application servers

Monitoring and Managing of application servers.
Introduction:
The Application server monitoring capability helps to analyze system services and resources. By
collecting and storing the analyzed data at regular intervals, the server memory, disk utilization,
server processes and server load are tracked.
Monitoring Capabilities
Currently the following performance statistics are monitored and measured.
1. Operating system metrics -- These are OS-related statistics, for example, CPU
utilization, physical or virtual memory, disk utilization, and processor-queue lengths.
2. Java virtual machine metrics -- This is the heap information.
3. Application server metrics -- This is a collection of data of server processes through
Java management extensions
4. Delivers both historical and current server performance metrics, delivering insight into
the performance over a period of time.
The following graph is an example of soap call metrics and it is calculated from its historical data
dated between 14-June and 9-August for the server named PADMA.
The values for X axis is from the call count of the commands STATSPART,
SYSTEN_STATS,PROCESS,CURSOR,COMMIT,DELETE and SELECT.
Y axis value is a date and time.
The current graph can be achieved by selecting current date and time.
JMX is used to measure the values for the above metrics chart and JFreeChart is used to
implement the charting. The following sections will explain about the JMX and its implementation
on this metrics project.
JMX
Java Management Extensions (JMX) is a technology that supplies tools for managing and
monitoring applications, system objects, devices and service oriented networks. Those resources
are instrumented by java objects called MBeans (Managed Beans). These MBeans are registered
in a core managed object server, known as an MBean server that acts as a management agent
and can run on most devices enabled for the Java programming language.
The way the resources are instrumented is completely independent from the management
infrastructure
A JMX agent consists of an MBean server in which MBeans are registered, and a set of services
for handling MBeans. In this way, JMX agents directly control resources and make them available
to remote management applications.
JMX technology defines standard connectors (JMX connectors) that allow you to access JMX
agents from remote management applications. JMX connectors using different protocols provide
the same management interface. Hence a management application can manage resources
transparently, regardless of the communication protocol used.
JMX is based on 3-level architecture:
1. Instrumentation Level where the applications expose their management functionality as
MBeans. It is basically instrumentation of the resources. In our case we are exposing the soap
calls as CommandMBeans.
2. Agent Level which is a container of instrumented entities and a set of standard management
services. The agent is the core of a JMX implementation and an intermediary between the MBean
and the applications. In our application the GlobalStats is a JMXAgent.
3. Client level or Remote Management level encompassing various management applications
which enables to access to the MBeanServer through the Connectors and Adaptors. In our
application MeasureRecorder.java is a management client.
These details are extracted from
http://java.sun.com/j2se/1.5.0/docs/guide/jmx/overview/JMXoverviewTOC.html. Please refer this
link from Sun Microsystems as it has detailed description of JMX architecture.
The implementations of these three levels in our application are follows.
1. Instrumentation of Soap Calls
To implement the instrumentation of Soap Calls as CommandMBean, we must follow the
design patterns and interfaces defined in the specification “JSR 3, Java Management Extensions
Instrumentation and Agent Specification”.
The standard JMX ObjectName format is using the following pattern,
[domainName]:[property=value][,property=value],…
Here is an example for existing JVM’s ObjectName “HeapMemoryUsage” is as follows,
java.lang:Type=Memory,Attribute=HeapMemoryUsage.
This Memory is already instrumented as MemoryMXBean in java.lang.management.
The following screen shot of “HeapMemoryUsage” measure is from JMXClient JConsole,
The same way the Soap Calls are instrumented in
com.aspc.Admin.jmc.register.CommandMBean. And the com.aspc.Admin.jmx.register.Command
implements the CommandMBean.
Basic rule of instrumentation is the Interface name must end with xxxMBean and method names
start with getXXX() will represent the attribute. An attribute means a measure.
The following methods return the attribute values.
1. public long getCallCount() = representing CallCount
2. public long getTimeTaken() = representing TimeTaken
3. public long getMostRecent() = representing MostRecent
4. public long getMinRecent() = representing MinRecent
5. public long getMaxRecent() = representing MaxRecent
6. public long getAvgRecent() = representing AvgRecent
7. public String getMaxRecentCommand() = representing MaxRecentCommand
8. public void update() to sets the value for the above measures.
The following is how these measures(attributes) are formatted for each soap call.
1. There are seven Attributes=
1. AvgRecent
2. CallCount
3. MaxRecent
4. MaxRecentCommand
5. MinRecent
6. MostRecent
7. TimeTaken
2. These seven attributes/measures are comes under each soapcall,
Names = Call name is dynamic and it will be added at runtime. The following screen shot
will illustrate the names, and the calls made so far are registered under their
DBCommand name. Example, SELECT,STATSPART and SYSTEM_STATS.
3. The call names are comes under Type = “Commands“
4. Finally The “Commands” is comes under Domain= “com.aspc”
The measures for call name “Select” will be looks like this,
Domain=com.aspc,Type=Select are recorded in the following forms.
com.aspc:Type=Commands,Name=Select,Attribute=AvgRecent
com.aspc:Type=Commands,Name=Select,Attribute=CallCount
com.aspc:Type=Commands,Name=Select,Attribute=MaxRecent
com.aspc:Type=Commands,Name=Select,Attribute=MaxRecentCommand
com.aspc:Type=Commands,Name=Select,Attribute=MinRecent
com.aspc:Type=Commands,Name=Select,Attribute=MostRecent
com.aspc:Type=Commands,Name=Select,Attribute=AvgRecent
com.aspc:Type=Commands,Name=Select,Attribute=TimeTaken
The following screen shot is a graphical view of soapcalls measures.
2. Agent Level:
2.1 JMXAgent
The main component of a JMX agent is the MBean server. This is a core managed object
server in which MBeans are registered. JMX agents directly control resources and make them
available to remote management agents.
In our application GlobalStats.java is a JMX Agent which registers CommandMBean as
ObjectName “com.aspc:Type=commands”.
Once a soapCall has been made, it registers that soap call in the ObjectName as
com.aspc:Type=commands,Name=”%SoapCallName%”.
It starts updating the attribute values whenever the same call has been made. This is
achieved in the method GlobalStats.recordCommandBean(), It actually invokes the
com.aspc.Admin.jmx.register.Command.update() method
The logic is written in the com.aspc.Admin.jmx.register.Command.update() which
calculates and sets the value for each measures.
2.2 JMX Server
The following parameters need to start JMX server while starting application server.
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=%jmx.port%
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
We can also set authentication, for that it needs jmxremote.password file and
jmxremote.access files.
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.password.file=../jmxremote.password
-Dcom.sun.management.jmxremote.access.file=../jmxremote.access
1. The jmxremote.access file defines two primary roles: Example,
“monitorRole readonly
controlRole readwrite”
2. The jmxremote.password file should contain the following values and it should be read-only
and not accessable from every other user.
“monitorRole $password$
controlRole $password$”
The %jmx.port% and this server IP will be used to connect to JMXServer remotely.
3. Management/Client Level:
3.1 Measure Recorder PadJul07
The JMX server has been established and JMXAgent is started measuring values for
each soapcalls. Now we can able to see the soap call measures.
The existing JMX architecture does not contain interface to record the measured values.
So we cannot see the statistic report of what was running 10 days before. To come across this
situation, we have implemented a recorder to store these values. The recorder is basically JMX
client and it will record the measures for each minute in the database.
Since we got the recorder we need to tell the recorder to store only particular
measure/attribute value.
The following tables will tell the recorder to store only the entries what it has.
1. MatrixSource (Contains a list of remote server details like server IP address, JMX port and
authentication details of that server.)
2. MatrixType (Contains list of measure details, which are domain=, attribute=, type=, etc…)
The values will be stored as FieldArray (time:value) in MatrixData table.
MatrixTask.java is another program which starts up MeasureRecorder for each MatrixSource data
and then the MeasureRecorder connects to the server and keep on recording measures for the
attributes specified in the MatrixType table.
3.2. Charting
Since system got the measured values, it needs to be presented as chart.
CommandChartPlotter has been implemented to create various types of charts for the JMX
measured values.
The following tables will be used for charting from MatrixData.,
1.MatrixChart,
2.MatricChartTimeRange
The following charting data entry screen is created in MatrixChart class. It takes the Charting
Date for timeRange X axis, measured value for Y axis.
The StartDate can be altered to see what the measure was on that date from. ChartTimeRange is
another field which specifies the time range in the map. It can be 5min, 10min, 30min,1hr,
2hr,…..,1yr,…etc. For Example, If start date = “09 Aug 2007 15:00” and time range = 10 min,
then the X axis value is from 15:00 to 15:10.
The measured value on the Y axis, is from the source and type specified in the Data section.
It can able produce different type of charts which are listed below.
1. PIE
2. BAR
3. TIME_SERIES
4. CANDLE_STICK
5. STACKED_BAR