The Artima Developer Community
Sponsored Link

Java Community News
JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax

19 replies on 2 pages. Most recent reply: Jul 25, 2007 2:51 PM by David Saff

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 19 replies on 2 pages [ 1 2 | » ]
Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 20, 2007 4:58 PM
Reply to this message Reply
Summary
The latest version of the Java unit testing framework includes new a syntax for expressing assertions, and allows developers to implement assumptions and theories in tests.
Advertisement

The JUnit project released the 4.4 version of the popular Java unit testing framework. In the project's release notes, project leaders David Saff and Kent Beck note that:

JUnit is designed to efficiently capture developers' intentions about their code, and quickly check their code matches those intentions. Over the last year, we've been talking about what things developers would like to say about their code that have been difficult in the past, and how we can make them easier.

Among the features introduced in this version is a new assertion syntax originally developed by Joe Walnes. The new syntax has the form:

 assertThat([value], [matcher statement]);

This makes statements such as the following possible:

    assertThat(x, is(3));
    assertThat(x, is(not(4)));
    assertThat(responseString, either(containsString("color")).or(containsString("colour")));
    assertThat(myList, hasItem("3"));

The advantages of this feature, according to the JUnit 4.4 release notes are a more readable syntax, better error messages, and the ability to combine matchers for a particular condition.

The new assertion syntax also enables two additional JUnit 4.4 features: assumptions and theories. Of the former, Saff and Beck write that:

Ideally, the developer writing a test has control of all of the forces that might cause a test to fail. If this isn't immediately possible, making dependencies explicit can often improve a design. For example, if a test fails when run in a different locale than the developer intended, it can be fixed by explicitly passing a locale to the domain code.

However, sometimes this is not desirable or possible. It's good to be able to run a test against the code as it is currently written, implicit assumptions and all, or to write a test that exposes a known bug. For these situations, JUnit now includes the ability to express "assumptions:"

import static org.junit.Assume.*

    @Test public void filenameIncludesUsername() {
       assumeThat(File.separatorChar, is('/'));
       assertThat(new User("optimus").configFileName(), is("configfiles/optimus.cfg"));
    }

    @Test public void correctBehaviorWhenFilenameIsNull() {
       assumeTrue(bugFixed("13356"));  // bugFixed is not included in JUnit
       assertThat(parse(null), is(new NullDocument()));
    }

Beck and Saff also note that in the JUnit 4.4 release, a failed assumption will still let a test pass, but in future versions this behavior may change.

Noting that,

A test captures the intended behavior in one particular scenario. A theory captures some aspect of the intended behavior in possibly infinite numbers of potential scenarios.

JUnit 4.4 introduces "theories," a feature borrowed from the Popper project.

  @RunWith(Theories.class)
    public class UserTest {
      @DataPoint public static String GOOD_USERNAME = "optimus";
      @DataPoint public static String USERNAME_WITH_SLASH = "optimus/prime";

      @Theory public void filenameIncludesUsername(String username) {
        assumeThat(username, not(containsString("/")));
        assertThat(new User(username).configFileName(), containsString(username));
      }
    }
Defining general statements in this way can jog the developer's memory about other potential data points and tests, also allows automated tools to search for new, unexpected data points that expose bugs.

What do you think of the new JUnit 4.4 features?


Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 20, 2007 10:26 PM
Reply to this message Reply

assertThat(x, is(3));
assertThat(responseString, either(containsString("color")).or(containsString("colour")
assertThat(myList, hasItem("3"));


And how the heck is this is an improvement over


assertEquals(3, x);
assertTrue(responseString.contains("color") || responseString.contains("colour"));
assertTrue(myList.contains("3"));


Sheesh - all four of the examples are just as easy, and far less confusing, to implement in the old-fashioned plain old Java syntax. For example - is the new JUnit either/or syntax for responseString "or" or "xor"? In Java I know.

All this "improvement" is just an improvement for JUnit consultants and more confusion for programmers. Why should I have to learn a whole new syntax for Boolean operations when Java has a perfectly good one? There's a reason mathemeticians developed symbols, not English to represent Boolean ops.

Stick to Plain Old Java Syntax (POJS). It's called JUnit, not CoolNewSyntaxUnit.

Jörn Zaefferer

Posts: 22
Nickname: jzaefferer
Registered: Jul, 2007

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 21, 2007 7:21 AM
Reply to this message Reply
@Morgan: I assume that those changes aim to provide a fluent interface for JUnit. Martin Fowler provides some a good overview of that: http://www.martinfowler.com/bliki/FluentInterface.html

David Saff

Posts: 9
Nickname: dsaff
Registered: Nov, 2005

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 21, 2007 2:44 PM
Reply to this message Reply
>
> assertThat(x, is(3));
> assertThat(responseString,
> either(containsString("color")).or(containsString("colour")
>
> assertThat(myList, hasItem("3"));
>

>
> And how the heck is this is an improvement over
>
>
> assertEquals(3, x);
> assertTrue(responseString.contains("color") ||
> responseString.contains("colour"));
> assertTrue(myList.contains("3"));
>


The most immediate benefit is that the error messages from the second group are immediately useful. See more information at:

http://junit.sourceforge.net/doc/ReleaseNotes4.4.html

It would be great if we could extract such useful error messages while still using Java's native operators (this would be possible, for example, in LISP). With the current design of the Java language, however, we've decided to leave the trade-off up to individual test writers.

Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 21, 2007 5:15 PM
Reply to this message Reply
I hardly ever look at the error messages, just go to the code where it failed. Unless I wrote and still remember that test, I'll not have any idea where or why "colour" was expected and how "something else" was generated.

Or, most likely, the test used to work, and recently started failing. I'd want to look at / debug through the code thats getting called to see what changed recently.

David Saff

Posts: 9
Nickname: dsaff
Registered: Nov, 2005

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 21, 2007 9:59 PM
Reply to this message Reply
> I hardly ever look at the error messages, just go to the
> code where it failed. Unless I wrote and still remember
> that test, I'll not have any idea where or why "colour"
> was expected and how "something else" was generated.

Consider

assertThat(responseString, anyOf(containsString("color"), containsString("colour")));
// ==> failure message:
// java.lang.AssertionError: 
// Expected: (a string containing "color" or a string containing "colour")
//      got: "Please choose a font"



In my own experience, I've really appreciated seeing right in the error message what the value of responseString is, without having to open the debugger or execute the target code in my head.

Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 22, 2007 11:34 AM
Reply to this message Reply
Lets assume that when you see ""Please choose a font" instead of "Colour" you know exacty what is wrong. Which is possible (though, IMO, not all that likely). What are you going to do?

a) You remember that Fred wrote the text properties code, assign him the bug.
b) Fred jumps into the code, fixes the bug, verifies it by rerunning the unit test, commits.

Let's say you have No error message

a) You notice that the error is in TestTextPropertiesDialog, Fred wrote TextPropertiesDialog, so you assign him the bug.
b) just like in case 1.

I still don't see any benefit.


Can you elaborate on the later parts of the article (assume, theory, etc.) Those sounded more interesting and potentially useful to me.

David Saff

Posts: 9
Nickname: dsaff
Registered: Nov, 2005

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 22, 2007 1:52 PM
Reply to this message Reply
> Lets assume that when you see ""Please choose a font"
> instead of "Colour" you know exacty what is wrong. Which
> is possible (though, IMO, not all that likely). What are
> you going to do?
>
> a) You remember that Fred wrote the text properties code,
> assign him the bug.
> b) Fred jumps into the code, fixes the bug, verifies it by
> rerunning the unit test, commits.
>
> Let's say you have No error message
>
> a) You notice that the error is in
> TestTextPropertiesDialog, Fred wrote TextPropertiesDialog,
> so you assign him the bug.
> b) just like in case 1.
>
> I still don't see any benefit.

Let's assume you're Fred. Does that change the scenario?

Using the new syntax has improved my development experience.
If a text properties dialog fails, one error message can tell me if:

1) I'm returning a null String.
2) I've somehow misspelled "coulor".
3) I'm accidentally talking about fonts instead.

Getting this for free is worth learning and using a different syntax for me. I can easily imagine that it would not be worth it for some people, for which all of the old assert methods will still be supported forever.

> Can you elaborate on the later parts of the article
> (assume, theory, etc.) Those sounded more interesting and
> potentially useful to me.

Have you read the release notes at

http://junit.sourceforge.net/doc/ReleaseNotes4.4.html

and followed up the links from there? I'd be happy to answer specific questions after that.

Johan Husman

Posts: 2
Nickname: ztream
Registered: Jul, 2007

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 7:09 AM
Reply to this message Reply
If that is the case, why the strange nesting of calls? That is, instead of this:

assertThat(x, is(3));
assertThat(x, is(not(4)));
assertThat(responseString, either(containsString("color")).or(containsString("colour")));
assertThat(myList, hasItem("3"));


why not


assertThat(x).is(3);
assertThat(x).isNot(3);
assertThat(responseString).either.containsString("color").or.containsString("co lour");
assertThat(myList).hasItem(3);


Seems both more readable and more in line with common examples of fluid interfaces (and frameworks such as rspec).

Johan Husman

Posts: 2
Nickname: ztream
Registered: Jul, 2007

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 7:13 AM
Reply to this message Reply
Oops, that was in reference to the post about Fluid Interfaces,

> @Morgan: I assume that those changes aim to provide a fluent interface for JUnit. Martin Fowler provides some a good overview of that:
> http://www.martinfowler.com/bliki/FluentInterface.html

Michael Wiedmer

Posts: 15
Nickname: micks
Registered: Aug, 2002

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 9:46 AM
Reply to this message Reply
I can only side with Morgan.

- Even having read through the new hoola release notes, the thing doesn't give me anything new that couldn't have been expressed with what's already been there.
- It's also very doubtful that the new API is more meaningful. It might be to some, to others (including me), it's not.
- And MOST OF THE TIME, the person sifting/changing some code of released software is not the one who's originally written it. That's why you should provide your own explicit failure text rather than relying on generic default output provided by a library.

* There problem to me is not that JUnit has a new API as it still supports the old one. My problem is that it's got a new dependency. And with that, my project all of a sudden has a new dependency to some Hamcrest party, too!

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 10:36 AM
Reply to this message Reply
> Stick to Plain Old Java Syntax (POJS). It's called
> JUnit, not CoolNewSyntaxUnit.

I think Morgan's got a point here. If users are going to break with plain Java syntax, why not use a fully-featured language that supports scripting (e.g. Groovy). Personally, I think that makes more sense anyway. It seems to me the main advantage of using Java for test scripts is that nobody is forced to leave their Java cocoon.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 10:59 AM
Reply to this message Reply
> JUnit now includes the ability to express "assumptions"
:
> With this release, a failed assumption will lead to the
> test being marked as passing, regardless of what the
> code below the assumption may assert. In the future,
> this may change, and a failed assumption may lead to the test being ignored

Is this saying that a new feature is being introduced that currently does nothing and whose future behavior may become version dependent? It sounds like an odd strategy.

In addition, I'm wary of a feature that potentially may lead to tests being ignored. Unless a test is commented out, it would seem to me more logical that tests should either pass or fail.

Joshua Nichols

Posts: 1
Nickname: nichoj
Registered: Jul, 2007

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 12:33 PM
Reply to this message Reply
>

> assertThat(x).is(3);
> assertThat(x).isNot(3);
> assertThat(responseString).either.containsString("color").
> or.containsString("colour");
> assertThat(myList).hasItem(3);
>


While I agree this would read better and be more inline with fluent interfaces, I think it's trickier to implement.

Keep in mind, that is(), not(), etc, are all static imports. So, you could write your own and statically import them, and go off without much hassle.

To use assertThat(something).is(somethingelse), is() would have to be a method on an object that assertThat() returns. So to add your own methods, you'd have to make a subclass of whatever assertThat() might return, and you'd have to have a way to make sure that assertThat() is returning your new subclass. I can't even think about how you might go about achieving that

This seems like it would take away a lot of the flexibility of using static imports provides you.

David Beutel

Posts: 29
Nickname: jdb
Registered: May, 2003

Re: JUnit 4.4 Comes with a Few Assumptions, Theories, and Some New Syntax Posted: Jul 23, 2007 4:01 PM
Reply to this message Reply
> Lets assume that when you see ""Please choose a font"
> instead of "Colour" you know exacty what is wrong. Which
> is possible (though, IMO, not all that likely). What are
> you going to do?
> [...]
>
> I still don't see any benefit.

It's like including a detail message or cause when you throw an Exception. It often makes maintenance easier, especially for an intermittent problem or nondeterministic test.

This is not just an issue with the new API. I hate going to a test failure and finding something like:

assertTrue( foo == 42 );

instead of

assertEquals( 42, foo );

Why did the test fail? What was foo? Why should I need to go into the debugger just to find out what foo was? I always change these to assertEquals().

Someone else suggested using the message for this:

assertTrue( "foo should be 42 but was " + foo, foo == 42 );

but that is redundant code which increases the maintenance cost with no advantage over assertEquals().

The assertEquals() API is more appropriate and the better choice. I imagine that this new API will also be the better choice, for the same reason.

Flat View: This topic has 19 replies on 2 pages [ 1  2 | » ]
Topic: Testing GWT Ajax Applications Previous Topic   Next Topic Topic: Geert Bevin on New Features in RIFE 1.6


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us