|
|
Re: ScalaTest 0.9 Released
|
Posted: Jan 16, 2008 10:44 AM
|
|
> Hi Bill,
>
> Nice job with ScalaTest! I'll have to take a closer look,
> it will be interesting to see how testing paradigms adapt
> to Scala.
>
> I like the fact that the test discovery mechanism can be
> overridden simply, but it also scares me somewhat. I
> think it might be an example where less flexibility is
> more desirable, or you will end up with someone sending
> you a ScalaTest class example and you will have no idea
> which ones are test methods because they overrode that
> behavior...
>
> Obviously, I'm partial to the annotation approach because
> it lets you specify attributes... For example, how would
> you specify a test method with a time out?
>
I didn't support timeouts yet. The way you'd probably do that in Scala is with a control abstraction, like
withTimeout(1000) {
// do something
}
Then let that throw an assertion error that indicates the problem was a timeout in the report. A few things that you used annotations for in TestNG I think are smoother with control abstractions, because the annotations approach requires that you make a test method for each. Whereas with a control abstraction, you can put multiple timeout tests or multiple expected exception tests in the same test method:
def testSomething() {
expectNPE {
throw new NullPointerException
}
expectNPE {
"hi".contains(null)
}
withTimeout(1000) {
// do one thing
}
withTimeout(2000) {
// do something else
}
}
But another way they could do it is extend TestNGSuite or JUnit4Suite and use TestNG's or JUunit4's timeout annotation on a test method.
class MySuite extends TestNGSuite {
@Test(threadPoolSize = 3, invocationCount = 10, timeOut = 10000)
def testServer() {
// do something, but in here you can use ScalaTest-like assertion syntax
assert(a === b)
expect(classOf[ServerException]) {
// do something else
}
}
}
Basically I would like ScalaTest to make it easy to write TestNG or JUnit tests that then use the nice assertion and expected exception syntax made possible by Scala, as well as property-based tests a la ScalaCheck. That way people using JUnit or TestNG can continue to do so, but write the code in Scala in a style that's more natural to Scala.
Anyway, I didn't support timeouts because I have never needed them and didn't know what people really wanted. I figured I'd wait until people ask. For the time being, it is quite straigtforward for people who need it to make a withTimeout-like control abstraction of their own.
> Similarly, I need to think more about the fact that each
> class can override its own execute method. In the JUnit
> world, a lot of people seem to want custom runners to do
> all kinds of things, but very few people have made similar
> requests for TestNG because a lot of the features users
> ask for are already built in. In echo to my previous
> point, I would encourage you to think about a world where
> hundreds of ScalaTest classes can each have a different
> way to identify test methods *and* a different way to
> actually run...
>
Well I think that was the world I was envisioning. It is more an OO approach of asking an object to use its data in an interesting way rather than doing something to the object from the outside.
Also, "test" is more abstract than "test method". A test method is one kind of test you can report. But a test is just anything that has a name, can be started, will either succeed or fail, and can optionally be ignored. That's about it. So another way people may want to express tests in ScalaTest is as function values. You can see an example of that approach here:
http://rehersal.sourceforge.net/documentation.shtml#writingTests
Except I'd do it this way:
test("Example of a test that will fail") {
fail()
}
In this approach, test is a method that registers the function value, which then would get invoked when execute is invoked on the Suite. This would look foreign to JUnit users I think, but may feel more natural to functional programming folks. I want to have a Suite to facilitate this kind of testing too. I don't think anyone used Rehersal much, so I probably won't make a RehersalSuite. I'll probably call this FunctionalSuite or something.
Another way is wrapping existing JUnit/TestNG, etc., tests written in Java. Tests in such suites would be JUnit/TestNG test methods. Although those are also "test methods."
But it could be anything else too, really, so people have a lot of leeway to integrate existing Java test code into ScalaTest suites, so they can support the old tests in Java and write new ones in Scala.
> Another quick thought: I strongly recommend giving
> priority to an Eclipse plug-in over a Maven one. TestNG's
> popularity spiked really hard as soon as an Eclipse
> plug-in was made available. The Maven crowd will come to
> you once your tool becomes more popular, and trust me, you
> really don't want to get into Maven with an alpha version
> of your code...
>
OK. I have never used Maven. It just seemed like the next step after Ant. I'll definitely look into the plug ins, and also contact the folks making the Scala plug ins.
> Back to your question...
>
> To be honest, I thought I had a good reason to put the
> value first and expected second but I can't remember it
> now... Off the top of my head, I would say it's because
> putting expected first reminds me of
>
> if (null == foo)
>
> which is obsolete in Java and more often written
>
> if (foo == null)
>
Interesting. In Scala you can do either:
scala> null == "hi"
res0: Boolean = false
scala> "hi" == null
res1: Boolean = false
> Putting expected first also makes sense since it mimics
> the failure message, which is typically "Expected 'exp'
> but got 'val'".
>
> So all in all, I don't feel strongly either way.
>
OK. I'm definitely leaning towards changing it to assert(expected === actual) then.
> What I do remember is that I purposely moved the string
> diagnosis to the end of the parameter list.
>
Me too. I did this in SuiteRunner in 2001.
> I have never liked to see this string as first parameter
> in JUnit's assert methods because when I read the assert,
> this is the last thing I'm interested in. What I want to
> know first and foremost is the expected value (another
> minor argument in favor of putting the expected value
> first). Of course, one might argue that you'd want to see
> the value under test first in the assert...
>
Elvin Eklund mentioned in this forum that they thought it was more readable with expected first, because the expected expression was usually shorter (fewer characters). I looked in my tests and found that to be true. I think he may be right that having the usually shorter one first improves readability.
|
|
Previous Topic
