Sunday, February 24, 2013

Benchmarking With JUnitBenchmark

If you are interested in doing some benchmark with JUnit, there is a very useful library for that: JUnitBenchmark. It has really nice features like:
  • Recording of execution time (average and standard deviation)
  • Monitoring garbage collector activity
  • Benchmark warm-up

To demonstrate what can be done with it, let's go with a trivial example comparing 2 components that handle String concatenation:
  1. StringBuilder from the JDK
  2. TextBuilder from Javolution

The first thing is to add dependencies for JUnitBenchmark and Javolution into the pom.xml:
        
        
        
        
            junit
            junit
            4.11
            test
        
        
        
        
            com.carrotsearch
            junit-benchmarks
            0.5.0
            test
        
        
        
        
            org.javolution
            javolution
            5.3.1
            test
        

Then here is the code for the JUnit Benchmark:
package org.javabenchmark;

import com.carrotsearch.junitbenchmarks.AbstractBenchmark;
import com.carrotsearch.junitbenchmarks.BenchmarkOptions;
import javolution.text.TextBuilder;
import org.junit.Test;

/**
 * Benchmark for String concatenation. Compares StringBUilder (JDK) and
 * TextBuilder (Javolution).
 */
public class StringConcatenationBenchmark extends AbstractBenchmark {

    public static final long LOOPS_COUNT = 10000000;

    @Test
    @BenchmarkOptions(benchmarkRounds = 3, warmupRounds = 1)
    public void stringBuilderBenchmark()  {
        
        StringBuilder builder = new StringBuilder();
        for (long i = 0; i < LOOPS_COUNT; i++) {
            builder.append('i').append(i);
        }
        System.out.println(builder.toString().length());
    }
    
    @Test
    @BenchmarkOptions(benchmarkRounds = 3, warmupRounds = 1)
    public void textBuilderBenchmark()  {
        
        TextBuilder builder = new TextBuilder();
        for (long i = 0; i < LOOPS_COUNT; i++) {
            builder.append('i').append(i);
        }
        System.out.println(builder.toString().length());
    }
}


And the output produced by the test on my machine (Ubuntu VM with 5Go and Core i7):

  • StringConcatenationBenchmark.stringBuilderBenchmark: [measured 3 out of 4 rounds, threads: 1 (sequential)] round: 0.68 [+- 0.06], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 6, GC.time: 0.19, time.total: 2.66, time.warmup: 0.64, time.bench: 2.03
  • StringConcatenationBenchmark.textBuilderBenchmark: [measured 3 out of 4 rounds, threads: 1 (sequential)] round: 0.47 [+- 0.01], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 5, GC.time: 0.18, time.total: 1.97, time.warmup: 0.57, time.bench: 1.41

Conclusion:
This micro-benchmark demonstrated that TextBuilder is 30% faster than StringBuilder (in our trivial test) and more stable (+- 0.01 compared to +- 0.06). GC metrics are similar in both cases.

6 comments:

  1. Typos in the POM example: it should be , not (note the capital 'I').

    Great example, though. :)

    ReplyDelete
    Replies
    1. I think that some XML parts of your message are missing :)

      Delete
  2. the United States. There are a wide range of pen.io/ models to select from, ranging from fundamental Visit URL versions to versions furnished Garbage Disposal Product Review 2017 with every one of the most recent bells as.

    ReplyDelete
  3. the beaten track, as it pivots maximizing a lot more room. soup.io/ The zinc alloy bar has a hefty Visit here but smooth take care of activity. easy pieces: best budget kitchen faucets The springtime as well as spray electrical Best Kitchen Faucet Reviews outlet is run by one key control as well as has adjustable spray activities. Kitchen Faucet The selector switch offers either vapor or spray.

    ReplyDelete