Summary
The next version of ScalaTest will include a set of matchers. These matchers demonstrate the kind of DSL notation you can create in Scala. In this post, I give a sneak preview of ScalaTest matchers, and ask for feedback.

---

Advertisement

I wanted to give a sneak preview of ScalaTest matchers, and get some feedback on the syntax. I still have some work to do on them before releasing them, but should be releasing them in the next few weeks as part of ScalaTest version 0.9.5.

The next release of ScalaTest will include two new traits, ShouldMatchers and MustMatchers. These two traits are basically identical except where ShouldMatchers says should, MustMatchers says must. None of the suite traits will mix either of these in by default, for two reasons. One is that some people prefer "should," and others "must," and this way everyone can select the verb they prefer. Also, both matchers traits involve a lot of implicit conversions, and I prefer that people invite these conversion into their test code explicitly. In this blog post, I'll show ShouldMatchers examples.

Matchers represent a kind of domain specific language (DSL) for assertions, which read a bit more like natural language in the source code, and provide more detailed error messages in assertions when they fail. Some examples of matchers in other test frameworks are Hamcrest matchers (Java), RSpec matchers (Ruby), easyb's ensure syntax (Groovy), and specs matchers (Scala).

For example, for a basic equality comparison with ScalaTest matchers, you can say:

object should equal (3)

Here object is a variable, and can be of any type. If the object is an Int with the value 3, nothing will happen. Otherwise, an assertion error will be thrown with the detail message, such as "7 did not equal 3".

Checking Size and length

You can check for a size of length of just about any type of object for which it would make sense. Here's how checking for length looks:

object should have length (3)

This syntax can be used with any object that has a field or method named length or a method named getLength. (The Scala compiler will check for this at compile time.) Size is similar:

object should have size (10)

Checking strings

You can check for whether a string starts with, ends with, or includes a substring or regular expression, like this:

string should startWith substring ("Hello")
string should startWith regex ("Hel*o")
string should endWith substring ("world")
string should endWith regex ("wo.ld")
string should include substring ("seven")
string should include regex ("wo.ld")

You can check whether a string fully matches a regular expression, like this:

string should fullyMatch regex (decimal)

Greater and less than

You can check whether any type that either is, or can be implicitly converted to, an Ordered[T] is greater than, less than, greater than or equal, or less than or equal to a value of type T, like this:

one should be < (7)
one should be > (0)
one should be <= (7)
one should be >= (0)

Checking predicate methods

If an object has a method that takes no parameters and returns boolean, you can check it by placing a Symbol (after be) that specifies the name of the method (excluding an optional prefix of "is"). A symbol literal in Scala begins with a tick mark and ends at the first non-identifier character. Thus, 'empty results in a Symbol object at runtime, as does 'defined and 'file. Here's an example:

emptySet should be ('empty)

Given this code, ScalaTest will use reflection to look on the object referenced from emptySet for a method that takes no parameters and results in Boolean, with either the name empty or isEmpty. If found, it invokes that method. If the method returns true, nothing happens. But if it returns false, an assertion error will be thrown that will contain a detail message like:

Set(1, 2, 3) was not empty

This be syntax can be used with any type, as there's no way in Scala's type system to restrict it just to types that have an appropriate method. If the object does not have an appropriately named predicate method, you'll get an IllegalArgumentException at runtime with a detail message that explains the problem. (Such errors could be caught at compile time, however, with a compiler plug-in.)

If you think it reads better, you can optionally put a or an after be. For example, java.util.File has two predicate methods, isFile and isDirectory. Thus with a File object named temp, you could write:

temp should be a ('file)

Checking object identity

If you need to check that two references refer to the exact same object, you can write:

ref1 should be theSameInstanceAs (ref2)

Working with floating point numbers

To check wether a floating point number has a value that exactly matches another, you can use should equal:

sevenDotOh should equal (7.0)

Sometimes, however, you may want to check whether a floating point number is within a range. You can do that using be and plusOrMinus, like this:

sevenDotOh should be (6.9 plusOrMinus 0.2)

This expression will cause an assertion error to be thrown if the floating point value, sevenDotOh is outside the range 6.7 to 7.1.

Iterables, collections, sequences, and maps

You can use some of the syntax shown previously with Iterable and its subtypes. For example, you can check whether an Iterable is empty, like this:

iterable should be ('empty)

You can check the length of an Seq (Array, List, etc.), like this:

array should have length (3)
list should have length (9)

You can check the size of any Collection, like this:

map should have size (20)
set should have size (90)

In addition, you can check whether an Iterable contains a particular element, like this:

iterable should contain element ("five")

You can also check whether a Map contains a particular key, or value, like this:

map should contain key (1)
map should contain value ("Howdy")

Be as an equality comparison

All uses of be other than those shown previously work the same as if be were replaced by equals. This will be the only redundancy in the first release of ScalaTest matchers. It is there because it enables syntax that sounds more natural. For example, instead of writing:

result should equal (null)

You can write:

result should be (null)

(Hopefully you won't write that too much given null is error prone, and Option is usually a better, well, option.) Here are some other examples of be used for equality comparison:

sum should be (7.0)
boring should be (false)
fun should be (true)
list should be (Nil)

Being negative

If you wish to check the opposite of some condition, you can use not. However, when you use not, you must enclose the expression being negated in parentheses or curly braces. Here are a few examples:

object should not (be (null))
sum should not { be <= 10 }
mylist should not (equal (yourList))
string should not { startWith substring ("Hello") }

Combining matchers with and and/or or

You can also combine matcher expressions with and and or, however, you must usually place parentheses or curly braces around the larger (and or or) expression. Here are a few examples:

ten should { equal (2 * 5) and equal (12 - 2) }
one should { equal (999) or equal (2 - 1) }
one should { not (be >= 7) and equal (2 - 1) }

Working with Options

ScalaTest matchers has no special support for Options, but you can work with them quite easily using syntax shown previously. For example, if you wish to check whether an option is None, you can write any of:

option should equal (None)
option should be (None)
option should not { be ('defined) }

If you wish to check an option is defined, and holds a specific value, you can write either of:

option should equal (Some("hi"))
option should be (Some("ho"))

If you only wish to check that an option is defined, but don't care what it's value is, you can write:

option should be ('defined)

Those pesky parens

You may have noticed that I always put parentheses on the last token in the expressions I've shown. This not always required, but the rule is a bit subtle. If the number of tokens in the expression is odd, the parentheses are not needed. But if the number of tokens is even, the parentheses are required. As a result, I usually include them, because then there's no subtle rule to remember. In addition, although ScalaTest matchers doesn't define which value is "actual" and which "expected," I usually put the expected value last and I think wrapping it in parentheses emphasizes the expected value nicely. Nevertheless, you're free to leave them off in many cases, and you may feel it makes the code more readable. Here are some expressions that work without parentheses:

object should have length 3
object should have size 10
string should startWith substring "Hello"
string should startWith regex "Hel*o"
string should endWith substring "world"
string should endWith regex "wo.ld"
string should include substring "seven"
string should include regex "wo.ld"
string should fullyMatch regex decimal
one should be < 7
one should be > 0
one should be <= 7
one should be >= 0
temp should be a 'file
object1 should be theSameInstanceAs object2
iterable should contain element "five"
map should contain key 1
map should contain value "Howdy"

Features that were cut

I have probably taken out at least half or more of the syntax that I implemented, in an attempt to eliminate redundancy. It is easier to add things later than subtract them, so I wanted to start with a minimal syntax. One thing I removed was shouldNot, andNot, and orNot, which made it possible to be negative without (usually) providing parentheses. For example, instead of:

object should not { equal ("howdy") }

You could have written:

object shouldNot equal ("howdy")

Given that I suspect negative expressions (involving "not") are far rarer than positive expressions, and to eliminate redundancy given I needed the not syntax no matter what, I dropped this. It could be added later if users convince me it is helpful.

Another syntax I dropped was shouldThrow, because I felt it didn't add much over plain old intercept (from org.scalatest.Assertions), and in fact subtracted. Whereas intercept returns the caught exception to allow for further inspection, shouldThrow did not.

One other tradeoff is that although and, or, and not may produce more readable code than &&, || and !, one downside of using them is they all have the same precedence. Were I to use the corresponding operators, the precedence would work as expected. In other words "a or b and c results in "(a or b) and c," whereas "a || b && c" results in "a || (b && c)."

That's it for now. I'm looking for feedback, so please fire away in the forums topic for this blog post.

Talk Back!

Have an opinion? Readers have already posted 6 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Bill Venners adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Bill Venners is president of Artima, Inc., publisher of Artima Developer (www.artima.com). He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Active in the Jini Community since its inception, Bill led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard way to associate user interfaces to Jini services. Bill is also the lead developer and designer of ScalaTest, an open source testing tool for Scala and Java developers, and coauthor with Martin Odersky and Lex Spoon of the book, Programming in Scala.