Article Discussion
Next-Generation Testing with TestNG
Summary: TestNG is a Java unit testing framework that aims to overcome many limitations of JUnit. In this interview with Artima, TestNG creator Cédric Beust describes what JUnit deficiencies TestNG aims to solve, and talks about some common unit testing misconceptions, including the dangers of overt focus on obtaining complete test coverage. He also explains the benefits of data-driven testing, and provides a preview of some new TestNG features.
4 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: December 23, 2010 1:17 AM by Ruwan
    Frank
     
    Posts: 135 / Nickname: fsommers / Registered: January 19, 2002 7:24 AM
    Next-Generation Testing with TestNG
    August 21, 2006 9:00 PM      
    In this article, Cédric Beust describes his biggest JUnit frustrations, and how he addressed them in TestNG. What JUnit limitations are you unhappy with? And what do you think of TestNG's way of solving them?
    • Kenneth
       
      Posts: 2 / Nickname: kasajian / Registered: August 12, 2006 1:24 AM
      Re: Next-Generation Testing with TestNG
      August 26, 2006 7:47 PM      
      I stopped reading when the author basically says that what a lot of people consider to be unit-testing is not important, which is fine, but doesn't counter the other side's argument. I believe there is a benefit for making mock objects to improve speed. The author says it's not important, doesn't explain why other than the saying that he doesn't care.

      That's about when I stopped reading.
    • Rossen
       
      Posts: 1 / Nickname: rstoya05 / Registered: August 28, 2006 4:51 AM
      Re: Next-Generation Testing with TestNG
      August 28, 2006 10:23 AM      
      TestNG creator Cédric Beust contradicts himself on one of his main points. First he says "Unit testing is an important part of testing..." but further down claims that "It's not." The point he makes here is that JUnit tests should be allowed to make database calls and mock objects should be avoided.

      There is no argument here. JUnit does not preclude you from making database calls. In fact you should have both tests with database calls (integration testing) and tests using mock objects (logical unit testing). Each one of these serves a very different purpose. Vincent Massol covers this in good detail in his book "JUnit in Action". As for mock objects, beyond merely stubbing out expensive calls they can provide further benefits such testing interractions between the object under test and the classes it depends on.

      There are also a number inaccurate statements throughout this interview that should be pointed out for readers. For example Cédric Beust implies there is no way to initialize expensive objects in JUnit once around test classes or suites. In fact this can be done with the TestSetup class in the JUnit framework.

      As for not being able to swap tests in and out without recompiling that doesn't bother me. In the agile way of thinking all tests should run as often as possible, which is another reason to avoid expensive calls in logical unit testing.
      • Cedric
         
        Posts: 7 / Nickname: cbeust / Registered: February 27, 2004 3:21 AM
        Re: Next-Generation Testing with TestNG
        September 1, 2006 8:25 AM      
        Thanks for the comments.

        Yes, a few things didn't come the way I thought they would, sorry about that. Here are a few clarifications.

        Unit tests are important but I don't care about the arbitrary limitations that some people claim, such as "never access the network" or "never use files".

        Do whatever it takes to make sure your class works as expected, and most of the time, the running time won't suffer much and you'll get rapid feedback if something fails instead of having to rely on acceptance tests higher up the stack. Unless you want to run your tests every time you save a file, these times will not add up to enough to make a difference.

        As for mocks, they come at a price (designing a parallel hierarchy with its own semantics and maintaining it) and I've seen enough abuse of them that I strongly caution against them now. In my opinion, there is only one good reason to use a mock: when the object under test is hard or costly to materialize in a testing environment.
        • Ruwan
           
          Posts: 1 / Nickname: ruwan / Registered: December 22, 2010 7:12 PM
          Re: Next-Generation Testing with TestNG
          December 23, 2010 1:17 AM      
          Hi All,

          i'm actually doing QA and just want to see how i can use TestNG to do the functional and integration testing. Could i get some artifact regarding this if you have or could someone of you guide me in the regard.

          Thanks,
          Ruwan,