Thursday, April 11, 2013

Check that your code is thread-safe with JUnit and ContiPerf

This short article shows how to check that you write thread-safe code by transforming a JUnit test into a concurrent one with ContiPerf.

Suppose that you have to test a date formatter component that uses the famous SimpleDateFormat class, as illustrated below:
package org.javabenchmark;

import java.text.SimpleDateFormat;
import java.util.Date;

/**
 * Helper dedicated to format date in a standard way.
 */
public class NonThreadSafeDateFormatHelper {

    /**
     * the date format for standard representation.
     */
    private SimpleDateFormat standardDateFormat = new SimpleDateFormat("yyyy-MM-dd");

    /**
     * formats the given date using the standard date format: yyyy-MM-dd.
     *
     * @param date the date to format
     * @return a literal representation of the given date.
     */
    public String toStandardString(Date date) {
        return standardDateFormat.format(date);
    }
}

Then, you could write a junit test like this:
package org.javabenchmark;

import org.junit.Test;

import java.util.Calendar;

import static org.fest.assertions.api.Assertions.*;

public class NonThreadSafeDateFormatHelperTest {

    @Test
    public void shouldFormatRandomDate() {

        // random date
        Calendar calendar = Calendar.getInstance();
        calendar.set(Calendar.YEAR, (int) (1000 + Math.random() * 1000));
        calendar.set(Calendar.DAY_OF_YEAR, (int) (Math.random() * 365));

        // test
        NonThreadSafeDateFormatHelper dateFormatHelperToTest = new NonThreadSafeDateFormatHelper();
        String randomDateString = dateFormatHelperToTest.toStandardString(calendar.getTime());

        // general controls
        assertThat(randomDateString).isNotNull();
        assertThat(randomDateString).hasSize(10);
        // year control
        String literalYear = String.valueOf(calendar.get(Calendar.YEAR));
        assertThat(literalYear).isEqualTo(randomDateString.substring(0, 4));
        // month control
        String literalMonth = String.valueOf(calendar.get(Calendar.MONTH) + 1);
        if (literalMonth.length() == 1) {
            literalMonth = "0" + literalMonth;
        }
        assertThat(literalMonth).isEqualTo(randomDateString.substring(5, 7));
        // day control
        String literalDayh = String.valueOf(calendar.get(Calendar.DAY_OF_MONTH));
        if (literalDayh.length() == 1) {
            literalDayh = "0" + literalDayh;
        }
        assertThat(literalDayh).isEqualTo(randomDateString.substring(8));
    }
}

Next, running the test will produce the following output:
Testsuite: org.javabenchmark.NonThreadSafeDateFormatHelperTest
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.173 sec

The test passes, and you could think that everything is fine, but what will happen if the DateFormatHelper class is used in a concurrent way, for instance from a JSF page to display the current date ?

To check that your code can handle concurrency, you can modify the previous JUnit test like this:
package org.javabenchmark;

import org.junit.Rule;
import org.junit.Test;

import java.text.ParseException;
import java.util.Calendar;
import org.databene.contiperf.PerfTest;
import org.databene.contiperf.junit.ContiPerfRule;

import static org.fest.assertions.api.Assertions.assertThat;

public class NonThreadSafeDateFormatHelperPerfTest {

    @Rule
    public ContiPerfRule i = new ContiPerfRule();
    /**
     * the date format helper to test.
     */
    private NonThreadSafeDateFormatHelper dateFormatHelperToTest = new NonThreadSafeDateFormatHelper();

    @Test
    @PerfTest(invocations = 1000, threads = 2)
    public void shouldFormatRandomDatesConcurrently() throws ParseException {

        // random date
        Calendar calendar = Calendar.getInstance();
        calendar.set(Calendar.YEAR, (int) (1000 + Math.random() * 1013));
        calendar.set(Calendar.DAY_OF_YEAR, (int) (Math.random() * 365));
        
        // test
        String randomDateString = dateFormatHelperToTest.toStandardString(calendar.getTime());

        // general controls
        assertThat(randomDateString).isNotNull();
        assertThat(randomDateString).hasSize(10);
        // year control
        String literalYear = String.valueOf(calendar.get(Calendar.YEAR));
        assertThat(literalYear).isEqualTo(randomDateString.substring(0, 4));
        // month control
        String literalMonth = String.valueOf(calendar.get(Calendar.MONTH) + 1);
        if (literalMonth.length() == 1) {
            literalMonth = "0" + literalMonth;
        }
        assertThat(literalMonth).isEqualTo(randomDateString.substring(5, 7));
        // day control
        String literalDayh = String.valueOf(calendar.get(Calendar.DAY_OF_MONTH));
        if (literalDayh.length() == 1) {
            literalDayh = "0" + literalDayh;
        }
        assertThat(literalDayh).isEqualTo(randomDateString.substring(8));
    }
}

This is the same test, except that the test method is invoked 1000 times by 2 threads, producing this output:
Testsuite: org.javabenchmark.NonThreadSafeDateFormatHelperPerfTest
org.javabenchmark.NonThreadSafeDateFormatHelperPerfTest.shouldFormatRandomDatesConcurrently
samples: 999
max:     19
average: 0.04804804804804805
median:  0
Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.377 sec
So, there is now an error, indicating that the DateFormatHelper component is not thread-safe: do not let it go into production :)

Summary

Good unit tests are not sufficient when you are writing components that will evolve in multi-threaded environment, like web applications. You can easily check if your code is vulnerable to race condition with contiperf.

7 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. This is a cool idea, and I like the syntax a lot.

    The problem from my point of view: If you are smart enough to use a "is my code threadsafe test", you likely are smart enough to already write threadsafe code :)

    Plus the fact that some race conditions may only happen 1 time in 10 years.. you would need to run this test for a really long time to weed out all of your race conditions ;) Much better to just read up on JCIP and try to make thread safe code...

    ReplyDelete
    Replies
    1. Hi Brian, thanks for the comment. I agree with you but sometimes you test code that relies on customer framework, or third-party libraries. With this approach, you can spot concurrency issues that are hidden in the dependencies that your code is using without exploring source or javadoc to be sure that all component are thread-safe.

      Delete
  3. I agree with Brian,
    the title of this post could be "Check that your code is probably thread-safe with JUnit and ContiPerf".
    Regards.

    ReplyDelete
  4. Thanks! Does it work with TestNG?

    ReplyDelete
    Replies
    1. As far as i know, ContiPerf is designed for JUnit only.

      Delete
  5. Hi,

    I developed a tool called http://vmlens.com which can detect such bugs. It searches for memory fields which are accessed from different threads without synchronization. It will detect such bugs when at least two threads accessing a field without synchronization.

    Regards
    Thomas

    ReplyDelete