Monitor OS pauses with jHiccup when benchmarking Java App

Benchmarking java code implies to be sure that the underlying platform does not impact the performance of the JVM in any way. Hopefully, there is a tool that monitors pauses and stalls that may occur at platform level: jHiccup from Azul.

As explained on the Azul's site, jHiccup captures the aggregate effects of the Java Virtual Machine (JVM), operating system, hypervisor (if used) and hardware on application stalls and response time.

This tool records platform pause/stall into log files that you can import into a MS Excel file:

Next you can plot platform "hiccup" and percentiles as illustrated below:

chart from MS Excel

This chart shows that during a 25 seconds benchmark, 99% of the latencies collected are lesser than 300 milliseconds, with a peak of half a second ! Such values indicate that we are not in the case of a dedicated/quiet system, and there must be some background activities (actually, i ran this benchmark during a Windows 7 user session with mail, browser and other usual softwares).

As you can guess, such a tool is really useful to check:

• that your JVM is running on a dedicated or quiet system
• that application response time behavior are not influenced by the platform

Running jHiccup is quite simple, use the provided script to launch your JVM on Unix environment (e.g jHiccup /usr/bin/java <Java args> -jar UsefulProgram.jar -a -b -c ) or as an instrumentation agent in other cases (e.g -javaagent:D:\apps\jHiccup.1.2.2\bin\jHiccup.jar="-d 0 -i 1000 -l hiccuplog -c c.hiccuplog") and you are done !