The Artima Developer Community
Sponsored Link

Weblogs Forum
ScalaTest Matchers Preview

6 replies on 1 page. Most recent reply: Sep 17, 2010 9:56 AM by Søren Bjerregaard Vrist

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 6 replies on 1 page
Bill Venners

Posts: 2251
Nickname: bv
Registered: Jan, 2002

ScalaTest Matchers Preview (View in Weblogs)
Posted: Dec 30, 2008 8:45 PM
Reply to this message Reply
Summary
The next version of ScalaTest will include a set of matchers. These matchers demonstrate the kind of DSL notation you can create in Scala. In this post, I give a sneak preview of ScalaTest matchers, and ask for feedback.
Advertisement

I wanted to give a sneak preview of ScalaTest matchers, and get some feedback on the syntax. I still have some work to do on them before releasing them, but should be releasing them in the next few weeks as part of ScalaTest version 0.9.5.

The next release of ScalaTest will include two new traits, ShouldMatchers and MustMatchers. These two traits are basically identical except where ShouldMatchers says should, MustMatchers says must. None of the suite traits will mix either of these in by default, for two reasons. One is that some people prefer "should," and others "must," and this way everyone can select the verb they prefer. Also, both matchers traits involve a lot of implicit conversions, and I prefer that people invite these conversion into their test code explicitly. In this blog post, I'll show ShouldMatchers examples.

Matchers represent a kind of domain specific language (DSL) for assertions, which read a bit more like natural language in the source code, and provide more detailed error messages in assertions when they fail. Some examples of matchers in other test frameworks are Hamcrest matchers (Java), RSpec matchers (Ruby), easyb's ensure syntax (Groovy), and specs matchers (Scala).

For example, for a basic equality comparison with ScalaTest matchers, you can say:

object should equal (3)

Here object is a variable, and can be of any type. If the object is an Int with the value 3, nothing will happen. Otherwise, an assertion error will be thrown with the detail message, such as "7 did not equal 3".

Checking Size and length

You can check for a size of length of just about any type of object for which it would make sense. Here's how checking for length looks:

object should have length (3)

This syntax can be used with any object that has a field or method named length or a method named getLength. (The Scala compiler will check for this at compile time.) Size is similar:

object should have size (10)

Checking strings

You can check for whether a string starts with, ends with, or includes a substring or regular expression, like this:

string should startWith substring ("Hello")
string should startWith regex ("Hel*o")
string should endWith substring ("world")
string should endWith regex ("wo.ld")
string should include substring ("seven")
string should include regex ("wo.ld")

You can check whether a string fully matches a regular expression, like this:

string should fullyMatch regex (decimal)

Greater and less than

You can check whether any type that either is, or can be implicitly converted to, an Ordered[T] is greater than, less than, greater than or equal, or less than or equal to a value of type T, like this:

one should be < (7)
one should be > (0)
one should be <= (7)
one should be >= (0)

Checking predicate methods

If an object has a method that takes no parameters and returns boolean, you can check it by placing a Symbol (after be) that specifies the name of the method (excluding an optional prefix of "is"). A symbol literal in Scala begins with a tick mark and ends at the first non-identifier character. Thus, 'empty results in a Symbol object at runtime, as does 'defined and 'file. Here's an example:

emptySet should be ('empty)
Given this code, ScalaTest will use reflection to look on the object referenced from emptySet for a method that takes no parameters and results in Boolean, with either the name empty or isEmpty. If found, it invokes that method. If the method returns true, nothing happens. But if it returns false, an assertion error will be thrown that will contain a detail message like:
Set(1, 2, 3) was not empty

This be syntax can be used with any type, as there's no way in Scala's type system to restrict it just to types that have an appropriate method. If the object does not have an appropriately named predicate method, you'll get an IllegalArgumentException at runtime with a detail message that explains the problem. (Such errors could be caught at compile time, however, with a compiler plug-in.)

If you think it reads better, you can optionally put a or an after be. For example, java.util.File has two predicate methods, isFile and isDirectory. Thus with a File object named temp, you could write:

temp should be a ('file)

Checking object identity

If you need to check that two references refer to the exact same object, you can write:

ref1 should be theSameInstanceAs (ref2)

Working with floating point numbers

To check wether a floating point number has a value that exactly matches another, you can use should equal:

sevenDotOh should equal (7.0)

Sometimes, however, you may want to check whether a floating point number is within a range. You can do that using be and plusOrMinus, like this:

sevenDotOh should be (6.9 plusOrMinus 0.2)

This expression will cause an assertion error to be thrown if the floating point value, sevenDotOh is outside the range 6.7 to 7.1.

Iterables, collections, sequences, and maps

You can use some of the syntax shown previously with Iterable and its subtypes. For example, you can check whether an Iterable is empty, like this:

iterable should be ('empty)

You can check the length of an Seq (Array, List, etc.), like this:

array should have length (3)
list should have length (9)

You can check the size of any Collection, like this:

map should have size (20)
set should have size (90)

In addition, you can check whether an Iterable contains a particular element, like this:

iterable should contain element ("five")

You can also check whether a Map contains a particular key, or value, like this:

map should contain key (1)
map should contain value ("Howdy")

Be as an equality comparison

All uses of be other than those shown previously work the same as if be were replaced by equals. This will be the only redundancy in the first release of ScalaTest matchers. It is there because it enables syntax that sounds more natural. For example, instead of writing:

result should equal (null)

You can write:

result should be (null)

(Hopefully you won't write that too much given null is error prone, and Option is usually a better, well, option.) Here are some other examples of be used for equality comparison:

sum should be (7.0)
boring should be (false)
fun should be (true)
list should be (Nil)

Being negative

If you wish to check the opposite of some condition, you can use not. However, when you use not, you must enclose the expression being negated in parentheses or curly braces. Here are a few examples:

object should not (be (null))
sum should not { be <= 10 }
mylist should not (equal (yourList))
string should not { startWith substring ("Hello") }

Combining matchers with and and/or or

You can also combine matcher expressions with and and or, however, you must usually place parentheses or curly braces around the larger (and or or) expression. Here are a few examples:

ten should { equal (2 * 5) and equal (12 - 2) }
one should { equal (999) or equal (2 - 1) }
one should { not (be >= 7) and equal (2 - 1) }

Working with Options

ScalaTest matchers has no special support for Options, but you can work with them quite easily using syntax shown previously. For example, if you wish to check whether an option is None, you can write any of:

option should equal (None)
option should be (None)
option should not { be ('defined) }

If you wish to check an option is defined, and holds a specific value, you can write either of:

option should equal (Some("hi"))
option should be (Some("ho"))

If you only wish to check that an option is defined, but don't care what it's value is, you can write:

option should be ('defined)

Those pesky parens

You may have noticed that I always put parentheses on the last token in the expressions I've shown. This not always required, but the rule is a bit subtle. If the number of tokens in the expression is odd, the parentheses are not needed. But if the number of tokens is even, the parentheses are required. As a result, I usually include them, because then there's no subtle rule to remember. In addition, although ScalaTest matchers doesn't define which value is "actual" and which "expected," I usually put the expected value last and I think wrapping it in parentheses emphasizes the expected value nicely. Nevertheless, you're free to leave them off in many cases, and you may feel it makes the code more readable. Here are some expressions that work without parentheses:

object should have length 3
object should have size 10
string should startWith substring "Hello"
string should startWith regex "Hel*o"
string should endWith substring "world"
string should endWith regex "wo.ld"
string should include substring "seven"
string should include regex "wo.ld"
string should fullyMatch regex decimal
one should be < 7
one should be > 0
one should be <= 7
one should be >= 0
temp should be a 'file
object1 should be theSameInstanceAs object2
iterable should contain element "five"
map should contain key 1
map should contain value "Howdy"

Features that were cut

I have probably taken out at least half or more of the syntax that I implemented, in an attempt to eliminate redundancy. It is easier to add things later than subtract them, so I wanted to start with a minimal syntax. One thing I removed was shouldNot, andNot, and orNot, which made it possible to be negative without (usually) providing parentheses. For example, instead of:

object should not { equal ("howdy") }

You could have written:

object shouldNot equal ("howdy")

Given that I suspect negative expressions (involving "not") are far rarer than positive expressions, and to eliminate redundancy given I needed the not syntax no matter what, I dropped this. It could be added later if users convince me it is helpful.

Another syntax I dropped was shouldThrow, because I felt it didn't add much over plain old intercept (from org.scalatest.Assertions), and in fact subtracted. Whereas intercept returns the caught exception to allow for further inspection, shouldThrow did not.

One other tradeoff is that although and, or, and not may produce more readable code than &&, || and !, one downside of using them is they all have the same precedence. Were I to use the corresponding operators, the precedence would work as expected. In other words "a or b and c results in "(a or b) and c," whereas "a || b && c" results in "a || (b && c)."

That's it for now. I'm looking for feedback, so please fire away in the forums topic for this blog post.


Eirik Maus

Posts: 15
Nickname: eirikma
Registered: Oct, 2005

Re: ScalaTest Matchers Preview Posted: Jan 1, 2009 7:21 AM
Reply to this message Reply
Cool and more elegant than the corresponding Ruby Rspec features. Speaking of them: the first question that came up on a Rspec presentation I attended was this: "cool but useless, since the business people and the business domain require the requiements specifications to be in our native language (in this case norwegian), not in english. Is that possible?"

At least I know from my own work that the business domain entities are just not possible to translate to other languages. Their definition will too often rely on concepts that are not sufficiently similar (or even existing) in other countries or languages. And "camel-lang", with every second word in a different language (with different grammar...) is just too ugly to work outside the programming code.

Is it possible at some point not too far into the future to have language-specific wrappers around the core functionality?

I'd love to see some attempts at writing specs in a different language, even one quite similar to English grammar such as Dutch, German or French. My first guess is that any functionality put into filler words like "is (...ing)", "do", "be", "the", "will" creates problems as other languages often use other grammatical conventions to convey the same meaning using fewer words.

Bill Venners

Posts: 2251
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest Matchers Preview Posted: Jan 1, 2009 1:51 PM
Reply to this message Reply
I have tried to keep internationalization in mind as I wrote ScalaTest. One thing is the strings that show up in the ScalaTest GUI and reports come from a properties file, so that it can be localized to different languages. Right now I just have a properties file for English.

Another thing is that in ScalaTest you ask (or you ask the Runner to ask) a "suite" class to execute itself by invoking execute on it. That method decides how to run the tests. Suite's execute method uses reflection to look for methods with names that start with "test", like JUnit 3. FunSuite offers a test registration method, so that you can write tests as functions. Spec offers describe and it methods for registering descriptions and tests.

So another way to localize ScalaTest is to create Suite subtraits that translate these words. For German, for example, you could have a Spec that uses es instead of it. One way to do that is to copy the code of Spec, but if you don't mind having the English in there too, the simplest way is to just create a trait that extends Spec and defines an es method that turns around and invokes it in the supertrait, passing along all the arguments. I'm not sure how one would want to translate describe into German, but you could do that too in the same way.

Lastly, the matchers I showed in this blog post are also bundled up into two traits (in the trunk today, and in the next ScalaTest release), ShouldMatchers and MustMatchers. No suite traits mix these in by default. You have to mix them in explicitly. So you can select "should" or "must", whichever you prefer, but if someone wanted to make a trait of German matchers, they could call that "SollMatchers" or something, and you could mix that in instead. Making a matcher DSL is hard though. That one's quite a bit of work. German has a different word order so as you point out it isn't always as simple as just translating each token. But this way you could end up with tests and specs in any language.

ScalaTest was really designed for this kind of flexibility. It doesn't push an agenda, saying you have to write tests first or second, or have to use BDD terms, or English terms either. The execute method and the way things are bundled up in to traits to mix in means you can pretty much write tests however you prefer, and you can make the test code look however you want it to look.

Joshua Cough

Posts: 20
Nickname: joshcough
Registered: Jan, 2008

Re: ScalaTest Matchers Preview Posted: Jan 6, 2009 3:59 PM
Reply to this message Reply
I believe I've said this before, but I'll say it again. I think "should" leaves the door open that its possibly ok that a particular object isnt what it should be.

Reading: 6 should be 6 is confusing. To me it reads: 6 SHOULD BE 6, but....maybe its not, and maybe in certain situations its ok.

I believe we should all prefer MUST BE, in almost all situations.

6 must be 6

There is no ambiguity. There is no reason to ever think twice.

Bill Venners

Posts: 2251
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest Matchers Preview Posted: Jan 8, 2009 11:06 PM
Reply to this message Reply
> I believe I've said this before, but I'll say it again. I
> think "should" leaves the door open that its possibly ok
> that a particular object isnt what it should be.
>
> Reading: 6 should be 6 is confusing. To me it reads:
> 6 SHOULD BE 6, but....maybe its not, and maybe in certain
> situations its ok.
>
> I believe we should all prefer MUST BE, in almost all
> situations.
>
> 6 must be 6
>
> There is no ambiguity. There is no reason to ever think
> twice.

> I believe we should all prefer MUST BE, in almost all
> situations.
>
You mean we *must* all prefer MUST BE?

I agree "result should be 6" does hint that the result may not be 6, but not that it's OK if it's not. It may not be 6 because there may be a bug, or the functionality may not yet have been implemented (if you're writing tests first, for example). But that doesn't mean it's OK. That's just when you get an assertion.

I've met people who prefer should and people who prefer must. So I wanted users to be able to choose their preference. With ScalaTest matchers, if you mix in ShouldMatchers, you get to say should. If you mix in MustMatchers, you get to say must.

Joshua Cough

Posts: 20
Nickname: joshcough
Registered: Jan, 2008

Re: ScalaTest Matchers Preview Posted: Jan 9, 2009 10:52 AM
Reply to this message Reply
I don't think we all must favor Must Be... I just think we should.

It's rather trivial I suppose. But, if you're thinking of tests as your specification (which obviously ScalaTest has a lot of Spec stuff built in), then I think you should use the more direct "must".

I think it's great to have the option, however, I think that the examples should primarily use Must, since in most cases that is more direct. Then, we should say that you have the option to mix in ShouldMatchers.

Søren Bjerregaard Vrist

Posts: 1
Nickname: svrist
Registered: Sep, 2010

Re: ScalaTest Matchers Preview Posted: Sep 17, 2010 9:56 AM
Reply to this message Reply
Im having some troubles with the "contains element" part of this post.

The javadoc suggest that "element" should not be part of the expression:
http://www.scalatest.org/scaladoc/doc-1.2/org/scalatest/matchers/ShouldMatchers.html

Have I missed something?

Flat View: This topic has 6 replies on 1 page
Topic: Library or Framework? Previous Topic   Next Topic Topic: What I Want to Get Out of JavaOne 2010


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us