Download Java Performance Analysis 301

Java Performance Analysis 301
Peter Johnson ([email protected])
CMG 2006, December 7, Session 534, Paper 6033
Abstract
The Java Platform provides a variety of
mechanisms for monitoring the performance of
Java applications. There are several tools that are
freely available that can be used to monitor Java
applications. This paper describes some of those
tools, and the Java Management Extensions (JMX)
technology on which those tools are built.
Additionally, the paper shows how custom tools
can be built using JMX.
Java Performance Analysis 301
Dec. 7, 2006
Page 2
Outline
Review
101 – garbage collection analysis
201 – garbage collection variations
JVM monitoring tools:
JConsole
JStat
Java Management Extensions (JMX)
Monitoring an application server (JBoss)
Java Performance Analysis 301
Dec. 7, 2006
Page 3
Java Heap
Young Generation
Eden
Where new objects are created
Survivor spaces
Where garbage collector places objects that are still in use
Old Generation
Where tenured objects are placed
References
http://java.sun.com/docs/hotspot/gc1.4.2/
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
Java Performance Analysis 301
Dec. 7, 2006
Page 4
Minor Garbage Collection
Occurs when no room in eden for new object
Marks all reachable objects in eden and “from” survivor space
Copies those objects to “to” survivor space
Updates references accordingly
If “to” survivor space fills, overflows to old generation
Resets allocation pointer to start of eden
H
A
B
C
D
E
I
E
F
G
H
G
Old Generation
Eden
Java Performance Analysis 301
I
From
To From
To
Spaces
Dec. 7, 2006
Page 5
Major Garbage Collection
Occurs when insufficient room in old generation to hold to-betenured objects during a minor collection
Mark and Compact
Reachable objects are marked
Marked objects are moved to one end of the old generation
J
K M
L M
O N O
V W
A
B
C
D
P
Q
X
E
F
G
H
X
R
S
T
U
Old Generation
Eden
Java Performance Analysis 301
I
To From
Spaces
Dec. 7, 2006
Page 6
Garbage Collection Info Options
-verbose:gc
Heap usage before and after GC
Time spent during GC
-XX:+PrintGCDetails
Generation and heap size before and after GC
Time spent during GC
-XX:+PrintHeapAtGC
Generation sizes before and after GC
Space sizes and percent in use before and after GC
Memory locations of the heap, its generations, and their
spaces
Java Performance Analysis 301
Dec. 7, 2006
Page 7
Gathering GC Data
>java -verbose:gc my.app.Main
GC data printed
to stdout
...
[GC 1860K->1388K(1984K), 0.0005059 secs]
[GC 1900K->1446K(1984K), 0.0006679 secs]
[GC 1958K->1468K(2112K), 0.0006251 secs]
[Full GC 1468K-> 195K(2112K), 0.0131045 secs]
...
Analyzer
Simple program
w/ reg. expr.
Comma-separated
value (CSV) file
...
1860,1388,0.0005059
1900,1446,0.0006679
1958,1468,0.0006251
1468,195,0.0131045
...
Spreadsheet
Java Performance Analysis 301
Dec. 7, 2006
Page 8
Interpreting Graphed GC Data
Typical GC data graph
For many GCs, the heap slowly
fills as objects get moved to the
old generation
• Blue (size before GC) or
magenta (size after GC) lines
with positive slope
Then a full GC happens
• Drop in blue (or magenta) lines
• Yellow (GC time) dot in higher
position
Java Performance Analysis 301
Dec. 7, 2006
Page 9
Parallel and Concurrent Collectors
Parallel collector – young generation only
-XX:+UseParallelGC
-XX:ParallelGCThreads=number
By default, number of threads = number of CPUs
JVM 5.0 – on by default if >= 2CPUs and >= 1GB RAM
Concurrent Mark Sweep collector – tenured gen only
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC (parallel young gen collector)
Concurrent collection runs in phases
Some phases stop the application threads
Some phases run concurrently with application
threads
Java Performance Analysis 301
Dec. 7, 2006
Page 10
Concurrent Collector Phases
Concurrent GC thread
time
Initial marking phase
Locates all objects
directly referenced
by a root
• local variable
• static field
Application threads
Java Performance Analysis 301
Dec. 7, 2006
Page 11
Concurrent Collector Phases
Concurrent GC thread
time
Initial marking phase
Concurrent marking
phase
• Locates and
marks all objects
referenced
• All new objects
marked
• All changed
objects noted
Application threads
Java Performance Analysis 301
Dec. 7, 2006
Page 12
Concurrent Collector Phases
time
Concurrent GC thread
Initial marking phase
Concurrent marking
phase
Final marking phase
• Rechecks all
changed objects
• When done, all
live objects are
marked
Application threads
Java Performance Analysis 301
Dec. 7, 2006
Page 13
Concurrent Collector Phases
time
Concurrent GC thread
Initial marking phase
Concurrent marking
phase
Final marking phase
Concurrent
sweeping phase
• Frees memory
used by dead
objects
• Maintains list of
available memory
Application threads
Java Performance Analysis 301
Dec. 7, 2006
Page 14
Concurrent Collector Phases
time
Concurrent GC thread
Initial marking phase
Concurrent marking
phase
Final marking phase
Concurrent
sweeping phase
Reset phase
• Post-collection
clean up
Application threads
Java Performance Analysis 301
Dec. 7, 2006
Page 15
Outline
Review
101 – garbage collection analysis
201 – garbage collection variations
JVM monitoring tools:
JConsole
JStat
Java Management Extensions (JMX)
Monitoring an application server (JBoss)
Java Performance Analysis 301
Dec. 7, 2006
Page 16
JConsole
JVM 5.0
Monitors:
Threads
Heap usage
Classes
MBeans
Java Performance Analysis 301
Dec. 7, 2006
Page 17
JConsole
JVM 5.0
Monitors:
Threads
Heap usage
Classes
MBeans
Java Performance Analysis 301
Dec. 7, 2006
Page 18
JConsole
JVM 5.0
Monitors:
Threads
Heap usage
Classes
MBeans
Java Performance Analysis 301
Dec. 7, 2006
Page 19
JConsole
JVM 5.0
Monitors:
Threads
Heap usage
Classes
MBeans
Java Performance Analysis 301
Dec. 7, 2006
Page 20
JConsole
JVM 5.0
Monitors:
Threads
Heap usage
Classes
MBeans
Java Performance Analysis 301
Dec. 7, 2006
Page 21
JConsole – Memory Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 22
JConsole – Memory Tab
Blue line
outlines
selected
heap
Java Performance Analysis 301
Dec. 7, 2006
Page 23
JConsole – Memory Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 24
JConsole – Thread Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 25
JConsole – Classes Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 26
JConsole – MBeans Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 27
JConsole – MBeans Tab
Blue values are editable
Double-click
to expand
bold values
Java Performance Analysis 301
Dec. 7, 2006
Page 28
JConsole – MBeans Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 29
JConsole – MBeans Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 30
JConsole – MBeans Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 31
JConsole – VM Tab
Java Performance Analysis 301
Dec. 7, 2006
Page 32
Running the JConsole
Two steps:
1: Start Java application with these options:
Required, only option
needed to monitor
local application
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=<port>
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
Use all four options to monitor
a remote application. Note that
an unsecure connection is used.
You can also monitor using a secure connection
Java Performance Analysis 301
Dec. 7, 2006
Page 33
Running the JConsole
2: Start JConsole and connect to the JVM
Started JBoss with all four options,
with port 9999. Can connect using
either method.
Can supply connection options
on JConsole command line.
service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi
Java Performance Analysis 301
Dec. 7, 2006
Page 34
The jstat utility
Command line utility that provides statistics for
Garbage collection
Just-in-time compiler
Class loading/unloading
Which statistic
to report
Java application
process id
jstat statistic option process [wait count]
Report output
option
Number of times to get
statistics and time to
wait between retrievals
Java Performance Analysis 301
Dec. 7, 2006
Page 35
Statistic and Output Options
Heap statistics
-gc, -gccapacity, -gcnew, -gcutil, and many more...
Compiler statistics
-compiler, -printcompilation
Class loader statistics
-class
Output options
-hn - display column header every n lines
-t - display time since JVM started in column 1
http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/jstat.html
Java Performance Analysis 301
Dec. 7, 2006
Page 36
Jstat Output
jstat –gcutil –t 2345 1s 200
Process id
Seconds
since start
of app
Timestamp
8520.7
8521.7
8522.8
8523.7
Permanent
space usage %
Eden
usage %
S0
11.60
0.00
0.00
0.00
S1
0.00
28.81
17.39
29.52
Survivor
space
usage %
Repeat 200 times, once a second
E
O
84.88
45.07
82.71
0.00
31.96
31.96
31.97
31.97
P
99.97
99.97
99.97
99.97
Tenured gen
usage %
YGC
424
427
429
433
Time spent in
collections
YGCT FGC FGCT
2.065
2.074
2.087
2.104
6
6
6
6
1.463
1.463
1.463
1.463
GCT
3.528
3.537
3.550
3.567
# minor/full
collections
Java Performance Analysis 301
Dec. 7, 2006
Page 37
Outline
Review
101 – garbage collection analysis
201 – garbage collection variations
JVM monitoring tools:
JConsole
JStat
Java Management Extensions (JMX)
Managed Bean (MBean)
Monitoring an application server (JBoss)
Java Performance Analysis 301
Dec. 7, 2006
Page 38
Java Management Extensions (JMX)
Standard specification for management info
Statistics, operations, events
Java Specification Request 3 (JSR-3)
Ask server for for connector which
provides an MBean connection
Provide
management
capabilities
Server
Client
JMXConnector
MBeanServerConnection
MBean
MBean
MBean
Uses connection
to interact with
MBeans
Java Performance Analysis 301
You can
write
your own
MBeans
Dec. 7, 2006
Page 39
JMX Example Program
This example queries an MBean provided by the JVM.
public static void main(String[] args) throws Exception {
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
JMXConnector jconn = JMXConnectorFactory.connect(url);
MBeanServerConnection mconn =
jconn.getMBeanServerConnection();
ObjectName name = new ObjectName
("java.lang:type=GarbageCollector,name=Copy");
Object val = mconn.getAttribute(name,"CollectionCount");
System.out.println(name + "\n\tCollectionCount=" + val);
}
Java Performance Analysis 301
Dec. 7, 2006
Page 40
JMX Example Program
This example queries an MBean provided by the JVM.
public static void main(String[] args) throws Exception {
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
JMXConnector jconn = JMXConnectorFactory.connect(url);
MBeanServerConnection mconn =
jconn.getMBeanServerConnection();
URL usedname
to connect
to the
ObjectName
= new
ObjectName
server.
Port
9999
matches
("java.lang:type=GarbageCollector,name=Copy");
earlier JConsole example.
Object
val = mconn.getAttribute(name,"CollectionCount");
System.out.println(name + "\n\tCollectionCount=" + val);
}
Java Performance Analysis 301
Dec. 7, 2006
Page 41
JMX Example Program
This example queries an MBean provided by the JVM.
public static void main(String[] args) throws Exception {
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
JMXConnector jconn = JMXConnectorFactory.connect(url);
MBeanServerConnection mconn =
jconn.getMBeanServerConnection();
ObjectName name = new ObjectName
("java.lang:type=GarbageCollector,name=Copy");
Object val = mconn.getAttribute(name,"CollectionCount");
Get the JMX connector. For +
there
get
System.out.println(name
"\n\tCollectionCount="
+ val);
the MbeanServerConnection.
}
Java Performance Analysis 301
Dec. 7, 2006
Page 42
JMX Example Program
This example queries an MBean provided by the JVM.
public static void main(String[] args) throws Exception {
JMXServiceURL
url
= new JMXServiceURL(
Each MBean has
a multi-part
name. This
code converts the text form of the name
"service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
intojconn
the internal
form.
JMXConnector
= JMXConnectorFactory.connect(url);
MBeanServerConnection mconn =
jconn.getMBeanServerConnection();
ObjectName name = new ObjectName
("java.lang:type=GarbageCollector,name=Copy");
Object val = mconn.getAttribute(name,"CollectionCount");
System.out.println(name + "\n\tCollectionCount=" + val);
}
Domain
One or more property/value pairs, separated by commas
:
An object name consists of
Java Performance Analysis 301
Dec. 7, 2006
Page 43
JMX Example Program
This example queries an MBean provided by the JVM.
public static void main(String[] args) throws Exception {
JMXServiceURL url = new JMXServiceURL(
"service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
JMXConnector
jconn = JMXConnectorFactory.connect(url);
Get the value of an attribute and print it. Note
MBeanServerConnection
mconn
that we never get the
MBean=object.
jconn.getMBeanServerConnection();
ObjectName name = new ObjectName
("java.lang:type=GarbageCollector,name=Copy");
Object val = mconn.getAttribute(name,"CollectionCount");
System.out.println(name + "\n\tCollectionCount=" + val);
}
Java Performance Analysis 301
Dec. 7, 2006
Page 44
MBeanServerConnection Methods
queryNames
Gets names that match namespace & property criteria
getMBeanInfo
Find out what attributes, methods and events are supported
get/setAttribute(s)
Gets or sets one or more attribute values
add/removeNotificationListener
Register or unregister to be notified of certain events
create/unregisterMBean
Create or remove an Mbean
invoke
Call an operation on an Mbean
Java Performance Analysis 301
Dec. 7, 2006
Page 45
JBoss Application Server
JBoss Application Server is based on a JMX kernel*
JMX is used extensively
Diagram is from
The JBoss 4
Application Server
Guide, Chapter 2
* Up through 4.x, JBoss AS 5.0 uses an inversion-of-control microkernel
Java Performance Analysis 301
Dec. 7, 2006
Page 46
JMX-Console
http://localhost:8080/jmx-console
Java Performance Analysis 301
Dec. 7, 2006
Page 47
JMX-Console
Changeable
Attributes
Operations
Invokeable
Java Performance Analysis 301
Dec. 7, 2006
Page 48
Twiddle
Command line script for JBoss Application Server
Used to interact with MBean Server Connection
twiddle opt command arguments
Arguments vary based
on the command
The action to perform.
Ex: get, settattrs, invoke
Command options
Ex: -s <host>
http://docs.jboss.org/jbossas/jboss4guide/r2/html/ch2.chapter.html
Java Performance Analysis 301
Dec. 7, 2006
Page 49
Twiddle Example
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 50
Twiddle Example
The host running
the application server
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 51
Twiddle Example
Get the value of one
attribute from one MBean
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 52
Twiddle Example
The name of the MBean.
In this example, a database connection pool.
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 53
Twiddle Example
The name of the attribute
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 54
Twiddle Example
twiddle –s somehostname get
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool"
InUseConnectionCount
Result:
InUseConnectionCount=4
NOTE: Twiddle command must be all on one command line.
Java Performance Analysis 301
Dec. 7, 2006
Page 55
JMX Example Application for JBoss AS
public static void main(String[] args) throws Exception {
Hashtable env = new Hashtable();
String factory="org.jnp.interfaces.NamingContextFactory";
env.put(Context.INITIAL_CONTEXT_FACTORY, factory);
String url1 = "jnp://localhost:1099";
env.put(Context.PROVIDER_URL, url1);
Context ctx = new InitialContext(env);
MBeanServerConnection mconn =(MBeanServerConnection)
ctx.lookup("jmx/invoker/RMIAdaptor");
ObjectName name = new ObjectName(
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool");
Object val=mconn.getAttribute(name,"InUseConnectionCount");
System.out.println(name+"\n\tInUseConnectionCount="+val);
}
Java Performance Analysis 301
Dec. 7, 2006
Page 56
JMX Example Application for JBoss AS
public static void main(String[] args) throws Exception {
Hashtable env = new Hashtable();
String factory="org.jnp.interfaces.NamingContextFactory";
env.put(Context.INITIAL_CONTEXT_FACTORY, factory);
String url1 = "jnp://localhost:1099";
env.put(Context.PROVIDER_URL, url1);
Context ctx = new InitialContext(env);
MBeanServerConnection mconn =(MBeanServerConnection)
ctx.lookup("jmx/invoker/RMIAdaptor");
ObjectName name = new ObjectName(
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool");
Object val=mconn.getAttribute(name,"InUseConnectionCount");
System.out.println(name+"\n\tInUseConnectionCount="+val);
The Java Naming and Directory Interface (JNDI) is
}
used to look up the MBean Server Connection.
This is boilerplate, only line that changes is the URL
for the server (localhost).
Java Performance Analysis 301
Dec. 7, 2006
Page 57
JMX Example Application for JBoss AS
public static void main(String[] args) throws Exception {
Hashtable env = new Hashtable();
Rest of the code is same as from before,
String factory="org.jnp.interfaces.NamingContextFactory";
except for the MBean name & attribute.
env.put(Context.INITIAL_CONTEXT_FACTORY,
factory);
String url1 = "jnp://localhost:1099";
env.put(Context.PROVIDER_URL, url1);
Context ctx = new InitialContext(env);
MBeanServerConnection mconn =(MBeanServerConnection)
ctx.lookup("jmx/invoker/RMIAdaptor");
ObjectName name = new ObjectName(
"jboss.jca:name=jdbc/TheDS,service=ManagedConnectionPool");
Object val=mconn.getAttribute(name,"InUseConnectionCount");
System.out.println(name+"\n\tInUseConnectionCount="+val);
}
Java Performance Analysis 301
Dec. 7, 2006
Page 58
Interesting MBeans
jboss.jca:service=ManagedConnectionPool,name=<name>
Number of database connections (available, in use, max, min)
jboss.management.local:J2EEServer=local,j2eeType=<beantype>,J2EEApplication=<ear-file>,EJBModule=<jarfile>,name=<ejb-name>
-orjboss.management.local:J2EEServer=local,j2eeType=servlet,J2E
EApplication=<ear-name>,WebModule=<warname>,name=<servlet-name>
Stats – method hit counts and timings (min, max, average)
Important: see the white paper on using Stats with twiddle!
jboss.web:type=ThreadPool,name=http-<ip-address>-<port>
Thread counts, aka client connections (min, max, spare, current)
And many, many more...
Java Performance Analysis 301
Dec. 7, 2006
Page 59
Conclusions
You should now know how to
Use JConsole to monitor a variety of JVM data
Use JStat to monitor JVM heap and class statistics
Use JMX (via key MBeans) to monitor Java applications,
including application servers
Java Performance Analysis 301
Dec. 7, 2006
Page 60
Thank you!
Questions?
Java Performance Analysis 301
10/20/06
Page 61