Thursday, March 7, 2013

Continuous Performance and JUnit with ContiPerf

If you want to introduce some continuous performance into your Java project, you should definitively check the ContiPerf library : that tool transforms your unit tests into performance tests simply by adding few annotations !

For instance, suppose your need to do some XML serialization with JAXB as illustrated below and therefore you have to test the UserSerializer class.


The corresponding code for the UserSerializer class is:
package org.javabenchmark;

import java.io.File;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;

public class UserSerializer {

    public void serializeUserToXmlFile(User user, File file) throws JAXBException {

        JAXBContext jaxbContext = JAXBContext.newInstance(User.class);
        Marshaller jaxbMarshaller = jaxbContext.createMarshaller();
        jaxbMarshaller.marshal(user, file);
    }
}
and the corresponding code for the UserSerializer test would be:
package org.javabenchmark;

import java.io.File;
import java.util.UUID;
import javax.xml.bind.JAXBException;
import org.junit.Test;
import static org.fest.assertions.api.Assertions.*;

public class UserSerializerTest {
    
    @Test
    public void shouldSerializeRandomUser() throws JAXBException {
        
        // given a random user
        String id = UUID.randomUUID().toString();
        String firstName = "Julien";
        String lastName = "Paoletti";
        
        User randomUser = new User();
        randomUser.setId(id);
        randomUser.setFirstName(firstName);
        randomUser.setLastName(lastName);
        
        // when serializing it to a random XML file
        File xmlFile = new File("target/" + id + ".xml");
        xmlFile.deleteOnExit();
        
        UserSerializer serializer = new UserSerializer();
        serializer.serializeUserToXmlFile(randomUser, xmlFile);
        
        // then the file contains the corresponding XML
        String validXmlContent = ""
                + ""+ firstName + ""
                + ""+ lastName + ""
                + "";
        assertThat(xmlFile).hasContent(validXmlContent);
    }
}

The test passes,  it is now time to transform this unit test into a performance one in 2 steps:
  1. add a JUnit rule from ContiPerf
  2. add a ContiPerf annotation to configure the execution of the performance test

public class UserSerializerTest {

    @Rule
    public ContiPerfRule i = new ContiPerfRule();
    
    @Test
    @PerfTest(invocations = 200, threads = 2)
    public void shouldSerializeRandomUser() throws JAXBException {

        // ...
    }
}
As you can see, the test will be exectuted 200 times by 2 threads. On my computer (Ubuntu VM 5Go 4 CPU), running the test produces the following output:

org.javabenchmark.UserSerializerTest.shouldSerializeRandomUser
samples: 200
max:     140
average: 6.79
median:  4

You would think that a simple XML serialization could be faster (almost 7 ms on average). To get some clue, let's profile the performance test with VisualVM :


Ouch, 95% of the execution time is taken by the JAXBContext method newInstance(). How to improve this ? A quick search on the internet shows that the JAXBContext is actually thread-safe, so it is not necessary to instantiate it each time.

The previous code can be improved:
public class UserSerializer {
    
    private static JAXBContext jaxbContext;
    
    static {
        try {
            jaxbContext = JAXBContext.newInstance(User.class);
        } catch (JAXBException ex) {
            // handles exception
            // ...
        }
    }

    public void serializeUserToXmlFile(User user, File file) throws JAXBException {
        Marshaller jaxbMarshaller = jaxbContext.createMarshaller();
        jaxbMarshaller.marshal(user, file);
    }
}

Now, running the test again produces the output:

org.javabenchmark.UserSerializerTest.shouldSerializeRandomUser
samples: 200
max:     133
average: 1.4
median:  0


It is much better and then we can add performance checks to our test. To achieve this, ContiPerf provides the @Required annotation:
  • @Required(throughput = 20) : Requires to have at least 20 test executions per second 
  • @Required(average = 50): Requires an average execution time of not more than 50 milliseconds
  • @Required(median = 45): Requires that 50% of all executions do not take longer than 45 milliseconds
  • @Required(max = 2000): Requires that no invocation takes more than 2000 milliseconds (2 seconds)
  • @Required(totalTime = 5000): Requires that the sum of all execution times is not more than 5000 milliseconds (5 seconds)
  • @Required(percentile90 = 3000): Requires that 90% of all executions do not take longer than 3000 milliseconds
  • @Required(percentile95 = 5000): Requires that 95% of all executions do not take longer than 5000 milliseconds
  • @Required(percentile99 = 10000): Requires that 99% of all executions do not take longer than 10000 milliseconds
  • @Required(percentiles = "66:200,96:500"): Requires that 66% of all executions do not take longer than 200 milliseconds and 96% of all executions do not take longer than 500 milliseconds

In our case, we could check that the average does not exceed 2 ms and that 95% of all executions do not take longer that 1 ms:
public class UserSerializerTest {

    @Rule
    public ContiPerfRule i = new ContiPerfRule();

    @Test
    @PerfTest(invocations = 200, threads = 2)
    @Required(average = 2, percentile95 = 1)
    public void shouldSerializeRandomUser() throws JAXBException {
        // ...
    }
}

Conclusion
To enable continuous performance in your test:
  1. write a JUnit test that passes
  2. transform it into a performance one
  3. ensure that performance is fine
  4. add your performance requirements

54 comments:

  1. ContiPerf link is pointing to blogger.com

    ReplyDelete
    Replies
    1. You are right, my mistake. The link is now fixed. Thanks.

      Delete
  2. This looks a great tool, didn't know about it before. By the way, I have also shared few unit testing best practices, let me know how do you find it..

    ReplyDelete
    Replies
    1. I agree with you, the practices you describe are fondamental. To facilitate/automate point 3, i would suggest to give a look at http://databene.org/feed4junit.html

      Delete
    2. I forgot to add that i recommend to write your test controls/assertions with https://github.com/alexruiz/fest-assert-2.x

      Delete
  3. Hi, Good staff.
    If you know, I try to use contiperf with ant to generate csv report but I can't get it.
    Would you help to do it, please, if you can?

    A.L

    ReplyDelete
  4. If you like fest assert, give a try to AssertJ, it's a fork of fest assert which is not active anymore.

    Here it is : https://github.com/joel-costigliola/assertj-core

    Cheers,

    Joel

    ReplyDelete
    Replies
    1. Hi Joel, thanks for the tip, i will give it a try.

      Delete
  5. Hi, seems to be great tool.
    Do you know if it is possible to integrate contiperf with CI to track/ display trends with avarage/maximum performance values (not only relay on single report)?

    ReplyDelete
  6. Hi ,Thanks for this tool. very useful. I would like to change the report file name every time.instead of overwritting an existing file. can anyone help me on this ?

    ReplyDelete
  7. Hi !

    When using Contiperf it produces out to (in our case ) stdout. Iwould like to remove that output. Does there exist a way to do that?

    Currenltly we get
    KeyLoggerTest.loggingLogHighThreadTest
    samples: 18404212
    max: 32
    average: 0.0010268844979616622
    median: 0

    but this is actually not interesting. All I want to know is if the test failed or not.

    ReplyDelete
  8. Thanks for sharing such informative article on Loadrunner Automation testing tool. This load testing tool will provide most precise information about the quality of software. Loadrunner training in Chennai

    ReplyDelete
  9. Very nice, i like the way you explained. I also wrote something on similar lines on Automated Testing of Java apps. Hope you would like it - http://bit.ly/1M47DeI

    ReplyDelete
  10. Hi, how to integrate contiPerf reports to jenkins ??? it's possible to save them on db ???

    ReplyDelete
  11. Thanks for sharing this unique and informative content which provided me the required information.
    oracle training in chennai

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. This comment has been removed by the author.

    ReplyDelete
  14. This blog is so nice to me. I will continue to come here again and again. Visit my link as well. Good luck
    http://www.jualobataborsiherbal.com/ obat aborsi
    http://caramenggugurkankandungan.info/ cara menggugurkan kandungan
    http://obataborsi59.com/ obat aborsi
    http://obataborsi59.com/cara-menggugurkan-kandungan-dengan-cepat-dan-aman/ cara menggugurkan kandungan
    http://obattelatdatangbulan.info/ obat telat datang bulan
    http://klinikobataborsi.com/ jual obat aborsi
    http://jualobatpenggugurkandungan.net/ obat penggugur kandungan
    http://tandatandakehamilan.net/ tanda tanda kehamilan
    http://tandatandakehamilan.net/cara-cepat-dan-selamat-menggugurkan-kandungan/ cara menggugurkan kandungan

    ReplyDelete
  15. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharng this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete

  16. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing..
    Android App Development Company

    ReplyDelete
  17. Nice information my sincere thanks for sharing this post and please continue to share this kind of post
    Software Testing Training in Chennai

    ReplyDelete
  18. The blog gave me idea to perform regression test on database My sincere Thanks for sharing this post and please continue to share this post before i read this blog i didn't have any knowledge about this but now i got some knowledge.
    softwaretesting training in Chennai

    ReplyDelete
  19. Really very nice blog information for this one and more technical skills are improve,i like that kind of post. Software Testing Training in Chennai | Selenium Training in Chennai

    ReplyDelete

  20. Nice it seems to be good post... It will get readers engagement on the article since readers engagement plays an vital role in every blog.i am expecting more updated posts from your hands.
    iOS App Development Company
    iOS App Development Company

    ReplyDelete
  21. great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
    web design Company
    web development Company
    web design Company in chennai
    web development Company in chennai
    web design Company in India
    web development Company in India

    ReplyDelete
  22. You have provided an nice article, Thank you very much for this one. And i hope this will be useful for many people.. and i am waiting for your next post keep on updating these kinds of knowledgeable things...
    Fitness SMS
    Fitness Text
    Salon SMS
    Salon Text
    Investor Relation SMS
    Investor Relation Text

    ReplyDelete
  23. it is really amazing...thanks for sharing....provide more useful information...
    Mobile app development company

    ReplyDelete
  24. great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
    web design Company
    web development Company
    web design Company in chennai
    web development Company in chennai
    web design Company in India
    web development Company in India

    ReplyDelete
  25. You have provided an nice article, Thank you very much for this one. And i hope this will be useful for many people.. and i am waiting for your next post keep on updating these kinds of knowledgeable things...
    Texting API
    Text message marketing
    Digital Mobile Marketing
    Sms API
    Sms marketing

    ReplyDelete
  26. These ways are very simple and very much useful, as a beginner level these helped me a lot thanks fore sharing these kinds of useful and knowledgeable information.
    PSD to Wordpress
    wordpress website development

    ReplyDelete
  27. This Content is Great in terms of research and I must appraciate your effort to highlight these points so every one should be aware of it.
    ||Happy Wheels Demo Game at happywheels.in||
    ||Play happy wheels game free at classic happy wheels game||
    ||Play online Happy Wheels demo game at happy wheels||
    ||Enjoy Fireboy and watergirl game online at fireboywatergirl.co||

    ReplyDelete
  28. This is very good content you share on this blog. it's very informative and provide me future related information.

    python training in chennai | python training in bangalore

    python online training | python training in pune

    python training in chennai

    ReplyDelete
  29. I really like the dear information you offer in your articles. I’m able to bookmark your site and show the kids check out up here generally. Im fairly positive theyre likely to be informed a great deal of new stuff here than anyone
    java training in chennai | java training in bangalore

    java online training | java training in pune

    ReplyDelete
  30. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  31. Greetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.

    AWS Training in Bangalore | Amazon Web Services Training in Bangalore

    Amazon Web Services Training in Pune | Best AWS Training in Pune

    AWS Online Training | Online AWS Certification Course - Gangboard

    ReplyDelete
  32. I’ve desired to post about something similar to this on one of my blogs and this has given me an idea. Cool Mat.
    python online training
    python training in OMR
    python training course in chennai

    ReplyDelete
  33. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
    Devops Training in pune

    ReplyDelete

  34. Hmm, it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I had written and say, I’m thoroughly enjoying your blog. I as well as an aspiring blog writer, but I’m still new to the whole thing. Do you have any recommendations for newbie blog writers? I’d appreciate it.

    AWS Interview Questions And Answers

    AWS Training in Bangalore | Amazon Web Services Training in Bangalore

    AWS Training in Pune | Best Amazon Web Services Training in Pune

    Amazon Web Services Training in Pune | Best AWS Training in Pune

    AWS Online Training | Online AWS Certification Course - Gangboard

    ReplyDelete
  35. Amazon has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. Amazon Web Services (AWS) is a secure cloud services platform, offering compute power, database storage, content delivery and other functionality to help businesses scale and grow.For more information visit aws online training

    ReplyDelete