JVM Troubleshooting
JVM Troubleshooting¶
This lists useful commands and options for troubleshooting performance or other issues with the JVM.
Find JVMs available to connect through jcmd¶
jcmd
This will list all the JVM processes it can find with their pids.
List all commands available via jcmd in the JVM¶
jcmd <pid> help
This will for the particular JVM output the commands available.
Get help for a particular command¶
jcmd <pid> help <cmd>
Capturing a heap dump from JVM¶
Using jcmd
jcmd <pid> GC.heap_dump filename=<file>
# To capture live objects
jcmd <pid> GC.heap_dump filename=<file> -all
Using jmap to capture a heap dump¶
Take a dump of all objects, not just live objects in the binary format. This does not force a full GC and may include dead objects.
jmap -dump:format=b,file=<filename.hprof> <pid>
Heap dump with forced full GC before dump:
jmap -dump:live,format=b,file=<filename.hprof> <pid>
https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jmap.html
Heap Histogram¶
Live objects without forcing a full GC.
jcmd <pid> GC.class_histogram
Includes dead objects.
jmap -histo <pid>
Forces a full gc before outputting the histogram.
jmap -histo:live <pid>
Dump Native Memory Summary¶
Must be enabled with JVM flag -XX:NativeMemoryTracking=summary
Dumps a summary of the usage of the native memory by the JVM.
jcmd <pid> VM.native_memory summary
You can also take a baseline and do a comparison:
jcmd <pid> VM.native_memory baseline
You can then compare the baseline to the current:
jcmd <pid> VM.native_memory summary.diff
Dump class stats (>=J8)¶
Must be enabled with JVM flag -XX:+UnlockDiagnosticVMOptions
jcmd <pid> GC.class_stats
Take thread dump¶
kill -3 <pid>
The output for kill -3 goes to the stdout for the app in question, therefore do not expect it to be output at the shell where you execute the command.
Or using jcmd
jcmd <pid> Thread.print
- jcmd will output to the shell where you execute the command and can be piped to a file.
- jcmd requires the JDK be installed it is not part of the JRE.
In Java 11+ the dump also includes how long the thread has been running as well as the CPU time for the thread. Example
"http-nio-8080-ClientPoller" #42 daemon prio=5 os_prio=31 cpu=7.57ms elapsed=102.11s tid=0x00007f7bfc97d800 nid=0x14103 runnable [0x000070000da89000] │
java.lang.Thread.State: RUNNABLE │
at sun.nio.ch.KQueue.poll(java.base@11.0.11/Native Method) │
at sun.nio.ch.KQueueSelectorImpl.doSelect(java.base@11.0.11/KQueueSelectorImpl.java:122) │
at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@11.0.11/SelectorImpl.java:124) │
- locked <0x00000007f8247760> (a sun.nio.ch.Util$2) │
- locked <0x00000007f8247608> (a sun.nio.ch.KQueueSelectorImpl) │
at sun.nio.ch.SelectorImpl.select(java.base@11.0.11/SelectorImpl.java:136) │
at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:709) │
at java.lang.Thread.run(java.base@11.0.11/Thread.java:829)
We can see here the cpu=<how much time spent> and elapsed=<time running>
How to work with a JRE only¶
- Without the JDK there are a lot of missing tools, such as jcmd etc. Its still possible to handle these using jattach, from the author of async-profiler
- https://github.com/jattach/jattach
- Download: https://github.com/jattach/jattach/releases
Supported commands:
load : load agent library
properties : print system properties
agentProperties : print agent properties
datadump : show heap and thread summary
threaddump : dump all stack traces (like jstack)
dumpheap : dump heap (like jmap)
inspectheap : heap histogram (like jmap -histo)
setflag : modify manageable VM flag
printflag : print VM flag
jcmd : execute jcmd command
jattach