Benchmarking With JUnitBenchmark

If you are interested in doing some benchmark with JUnit, there is a very useful library for that: JUnitBenchmark. It has really nice features like:

• Recording of execution time (average and standard deviation)
• Monitoring garbage collector activity
• Benchmark warm-up

To demonstrate what can be done with it, let's go with a trivial example comparing 2 components that handle String concatenation:

1. StringBuilder from the JDK
2. TextBuilder from Javolution

The first thing is to add dependencies for JUnitBenchmark and Javolution into the pom.xml:

        
        
        
        
            junit
            junit
            4.11
            test
        
        
        
        
            com.carrotsearch
            junit-benchmarks
            0.5.0
            test
        
        
        
        
            org.javolution
            javolution
            5.3.1
            test
        

Then here is the code for the JUnit Benchmark:

package org.javabenchmark;

import com.carrotsearch.junitbenchmarks.AbstractBenchmark;
import com.carrotsearch.junitbenchmarks.BenchmarkOptions;
import javolution.text.TextBuilder;
import org.junit.Test;

/**
 * Benchmark for String concatenation. Compares StringBUilder (JDK) and
 * TextBuilder (Javolution).
 */
public class StringConcatenationBenchmark extends AbstractBenchmark {

    public static final long LOOPS_COUNT = 10000000;

    @Test
    @BenchmarkOptions(benchmarkRounds = 3, warmupRounds = 1)
    public void stringBuilderBenchmark()  {
        
        StringBuilder builder = new StringBuilder();
        for (long i = 0; i < LOOPS_COUNT; i++) {
            builder.append('i').append(i);
        }
        System.out.println(builder.toString().length());
    }
    
    @Test
    @BenchmarkOptions(benchmarkRounds = 3, warmupRounds = 1)
    public void textBuilderBenchmark()  {
        
        TextBuilder builder = new TextBuilder();
        for (long i = 0; i < LOOPS_COUNT; i++) {
            builder.append('i').append(i);
        }
        System.out.println(builder.toString().length());
    }
}

And the output produced by the test on my machine (Ubuntu VM with 5Go and Core i7):

• StringConcatenationBenchmark.stringBuilderBenchmark: [measured 3 out of 4 rounds, threads: 1 (sequential)] round: 0.68 [+- 0.06], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 6, GC.time: 0.19, time.total: 2.66, time.warmup: 0.64, time.bench: 2.03

• StringConcatenationBenchmark.textBuilderBenchmark: [measured 3 out of 4 rounds, threads: 1 (sequential)] round: 0.47 [+- 0.01], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 5, GC.time: 0.18, time.total: 1.97, time.warmup: 0.57, time.bench: 1.41

Conclusion:
This micro-benchmark demonstrated that TextBuilder is 30% faster than StringBuilder (in our trivial test) and more stable (+- 0.01 compared to +- 0.06). GC metrics are similar in both cases.