Summary
Tonight I released a new version of ScalaTest, which contains a few changes to the way in which you express assertions and a Suite discovery feature. This release contains all the features that I plan to have in version 1.0, so it is ready for a few months of beta testing.

In the latest version of ScalaTest, I adjusted the default way to do assertion-based testing, and completed a Suite discovery mechanism for Runner.

In the discussion about the previous release of ScalaTest, and on other forums and mailing lists, the difficulty of remembering which parameter (to methods like JUnit's assertEquals) was expected and which actual was brought up. In the previous version, the === operator was defined such that actual was on the left, and expected on the right. This turned out to be the opposite order from JUnit, which I didn't like. Michael Feathers said that because === was a palindrome, it made him expect its semantics would be symmetric, but that the "Expected 2, but got 1" reporting made it asymmetric. Elizabeth Wiethoff pointed out that Python's message didn't care about the order of parameters, and didn't mention expected or actual values. It would just say, 2 != 1. Elizabeth also expressed her preference for the Python approach.

This sent me back to the drawing board. In the current release, === is symmetric. Here's an example:

val a = 1
val b = 2
assert(a === b)

The failure report for the above code will simply say that "1 did not equal 2". I also created an expect construct that looks like this:

val a = 4
val b = 3
expect(2) {
  a - b
}

In this example you indicate that you expect the value 2 to be the result of the code in the curly braces. The failure message from this code will read, "Expected 2, but got 1". In the expect construct, it is clear which value is expected. The expect construct works very well as expressions get longer, whereas the assert is most readable when the expressions are shorter. The expect construct also looks very much like the intercept construct, which checks that expected exception type is thrown. Here is a test method from ScalaTest's own tests that shows interceptions and expectations doing their thing:

  def testConstructor() {
    intercept(classOf[NullPointerException]) {
      new DiscoverySuite(null, Set(), false, loader)
    }
    intercept(classOf[NullPointerException]) {
      new DiscoverySuite("hi", null, false, loader)
    }
    intercept(classOf[NullPointerException]) {
      new DiscoverySuite(null, Set(), false, null)
    }
    expect(Nil) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set(), false)
    }
    expect(List("a.b.c.Hi")) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set("a.b.c.Hi"), true)
    }
    expect(List("a.b.c.d.Hi")) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set("a.b.c.d.Hi"), true)
    }
    expect(List("a.b.c.Hi")) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set("a.b.c.Hi"), false)
    }
    expect(Nil) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set("a.b.c"), false)
    }
    expect(Nil) {
      DiscoverySuite.nestedSuiteNames("a.b.c", Set("a.b.c.d.Hi"), false)
    }
  }

Here are some assertions from a different test method:

  def testSimpleNameForTest() {
    val s = new SuiteFriend(new Suite {})
    assert(s.simpleNameForTest("testThis") === "testThis")
    assert(s.simpleNameForTest("testThis(Reporter)") === "testThis")
    assert(s.simpleNameForTest("test(Reporter)") === "test")
    assert(s.simpleNameForTest("test") === "test")
  }

The other major change in this release is that I completed the Suite discovery feature. Suite discovery was implemented in SuiteRunner (the Java testing tool from which ScalaTest sprung), but back then we did discovery when you pushed a button in the GUI. I ended up never using that feature, and long ago realized it should be on the command line. I never added command-line support to SuiteRunner, but just did so to ScalaTest. You can now request Suites to be discovered on the runpath with the -m and -w command line arguments to Runner.

ScalaTest is now "feature complete" for the 1.0 release. I want to release it 1.0 around JavaOne week. Prior to that, I'm hoping to get feedback from real users to fix bugs and potentially make other breaking changes like I did this week. I also plan to work to integrate with ScalaCheck, JUnit, TestNG, and JUnit 4, and make a FuncSuite in which tests are represented by function values. If ScalaTest is up for all that, I figure it's ready for anything, and I'll release it 1.0. So please download ScalaTest and give it a try. Post feedback either in the forum topic for this blog post or in the ScalaTest Forum.

Talk Back!

Have an opinion? Readers have already posted 29 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Bill Venners adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Bill Venners is president of Artima, Inc., publisher of Artima Developer (www.artima.com). He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Active in the Jini Community since its inception, Bill led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard way to associate user interfaces to Jini services. Bill is also the lead developer and designer of ScalaTest, an open source testing tool for Scala and Java developers, and coauthor with Martin Odersky and Lex Spoon of the book, Programming in Scala.