Weblogs Forum - ScalaTest 0.9 Released

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
ScalaTest 0.9 Released

58 replies on 4 pages. Most recent reply: Jan 21, 2008 3:26 AM by Bill Venners

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 58 replies on 4 pages [ « | 1 2 3 4 | » ]

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 15, 2008 6:09 PM

> > Hey Bill,
> >
> > you really should mention in your post that assert(x
> ===
> > y) is not the same as assert(x == y), rather that it is
> > able to print both expected and actual values for
> failed
> > assertions. Thats really slick and convinced me to give
> > ScalaTest a try.
>
>
> That's nice. The only thing I'd offer is that the fact
> that === is palindrome seems to imply that the operator is
> symmetric.
>
Well, the only difference if you switch the sides is what is called actual and expected in the report. I could call these left and right I suppose.

> HUnit in Haskell uses @=? as an operator with
> expected on the left and actual on the
> right.
>
> It's kind of neat because it looks like a little
> placeholder for an equation with '?' holding the position
> of the thing we are questioning.
>
It may be hard for people to remember which side is the actual and which side is the expected, but it also isn't a show stopper. As soon as someone sees the code they'll realize they had them backwards. Maybe I should put actual (left operand) and expected (right operand) in the report to make it clearer. Though I should wait until it is found to really be a problem. I haven't had that problem yet myself. One way to remember them is that they are in alphabetical order from left to right.

The one mistake I've found myself making is forgetting to add the extra = sign. I'll say assert(a == b) instead of assert(a === b). But that hasn't happened too often and again is not really a show stopper.

Eivind Eklund

Posts: 49
Nickname: eeklund2
Registered: Jan, 2006

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 12:59 AM

> Well, the only difference if you switch the sides is what
> is called actual and expected in the report. I could call
> these left and right I suppose.

Might be better.

> > HUnit in Haskell uses @=? as an operator with
> > expected on the left and actual on the
> > right.
> >
> > It's kind of neat because it looks like a little
> > placeholder for an equation with '?' holding the
> position
> > of the thing we are questioning.
> >
> It may be hard for people to remember which side is the
> actual and which side is the expected, but it also isn't a
> show stopper. As soon as someone sees the code they'll
> realize they had them backwards. Maybe I should put actual
> (left operand) and expected (right operand) in the report
> to make it clearer.

I have found that my test code gets easier to read if I put the expected value first, because the expected value is more often short/regular than the value I test against. YMMV, just a thing to think about.

Eivind.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 4:34 AM

> > Well, the only difference if you switch the sides is
> what
> > is called actual and expected in the report. I could
> call
> > these left and right I suppose.
>
> Might be better.

Seems like it would introduce more ambiguity if the report said: left 3 doesn't match right 4.

> > > HUnit in Haskell uses @=? as an operator with
> > > expected on the left and actual on the
> > > right.
> > >
> > > It's kind of neat because it looks like a little
> > > placeholder for an equation with '?' holding the
> > position
> > > of the thing we are questioning.
> > >
> > It may be hard for people to remember which side is the
> > actual and which side is the expected, but it also isn't
> a
> > show stopper. As soon as someone sees the code they'll
> > realize they had them backwards. Maybe I should put
> actual
> > (left operand) and expected (right operand) in the
> report
> > to make it clearer.
>
> I have found that my test code gets easier to read if I
> put the expected value first, because the expected value
> is more often short/regular than the value I test against.
> YMMV, just a thing to think about.
>
> Eivind.

I've only run into a couple of UT frameworks that put actual first. Most put expected first. So, it also has the principle of least surprise working for it.

Simple equality comparison is symmetric. The reporting aspect makes this comparison asymmetric.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 4:42 AM

> It may be hard for people to remember which side is the
> actual and which side is the expected, but it also isn't a
> show stopper. As soon as someone sees the code they'll
> realize they had them backwards. Maybe I should put actual
> (left operand) and expected (right operand) in the report
> to make it clearer. Though I should wait until it is found
> to really be a problem. I haven't had that problem yet
> myself. One way to remember them is that they are in
> alphabetical order from left to right.
>
> The one mistake I've found myself making is forgetting to
> add the extra = sign. I'll say assert(a == b) instead of
> assert(a === b). But that hasn't happened too often and
> again is not really a show stopper.

If you do have a == b, what happens? Is it a silent error?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 5:46 AM

> > The one mistake I've found myself making is forgetting
> to
> > add the extra = sign. I'll say assert(a == b) instead
> of
> > assert(a === b). But that hasn't happened too often and
> > again is not really a show stopper.
>
> If you do have a == b, what happens? Is it a silent error?

>
No, just a regular assertion. If false, you get a report, but it doesn't tell you a and b, just that the assertion failed. It's like assertTrue(a == b) in JUnit 3.

Elizabeth Wiethoff

Posts: 89
Nickname: ewiethoff
Registered: Mar, 2005

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 6:08 AM

I'm accustomed to putting the expected value on the left if only because test failure messages can be confusing otherwise.

In Ruby, assert_equal('foo', 'bar') produces something like Expected "foo" but got "bar". On the other hand, Python failure messages don't seem to care much about order: assertEqual('foo', 'bar') produces AssertionError: 'foo' != 'bar'.

I notice in your Equalizer === docs that the actual value should go on the left and the expected value on the right. What is the wording of the Scala failure message produced by assert("foo" === "bar")?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 6:11 AM

> I have found that my test code gets easier to read if I
> put the expected value first, because the expected value
> is more often short/regular than the value I test against.
> YMMV, just a thing to think about.
>
I just looked at my ScalaTest tests (they are in the zip file), and it looks like the expected is usually shorter in my case too. So it makes sense it might be easier to read if expected comes first. Plus I want this to be comfortable to JUnit users.

It was really a 50/50 choice for me. I considered putting the string message first to be a mistake in JUnit, but not the order of actual/expected. That wasn't broken and didn't need fixing. I should have looked at the order in JUnit and followed that, but didn't think of it. I'll probably switch the order in the next release.

One thing I did notice was that TestNG puts actual before expected too. I'll send an email to Cedric and ask why he did that, and whether people have complained, etc.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 6:14 AM

> I'm accustomed to putting the expected value on the left
> if only because test failure messages can be confusing
> otherwise.
>
> In Ruby, assert_equal('foo', 'bar') produces
> something like Expected "foo" but got "bar".
> On the other hand, Python failure messages don't seem to
> care much about order:

assertEqual('foo',
> 'bar')

produces

AssertionError: 'foo' !=
> 'bar'

.
>
> I notice in your Equalizer === docs that the
> actual value should go on the left and the expected value
> on the right. What is the wording of the Scala failure
> message produced by assert("foo" === "bar")?
>
That's the clincher. If you put assert(1 === 2), you get the message, "expected 2, but got 1". I'll very likely switch the order in the next release so you say assert(expected === actual). Thanks for the feedback!

Elizabeth Wiethoff

Posts: 89
Nickname: ewiethoff
Registered: Mar, 2005

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 7:06 AM

You're welcome. Although I said I'm accustomed to putting an "expected" value on the left and an "actual" value on the right, I do prefer Python's system of having assertEqual(first, second) produce first != second upon failure.

For example, if I invent some sort of color class which considers similar colors equal, I want to write some tests which have no notion of "expected" and "actual." I would like assert(navy_blue === midnight_blue) to pass--and I would like assert(grape === salmon) to fail with a message such as grape does not equal salmon.

My math brain thinks of equality as symmetric. Therefore, I find an asymmetric failure message such as expected this but got that a bit disturbing.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 7:30 AM

> You're welcome. Although I said I'm accustomed to
> putting an "expected" value on the left and an "actual"
> value on the right, I do prefer Python's system of having
> assertEqual(first, second) produce
> first != second upon failure.
>
> For example, if I invent some sort of color
> class which considers similar colors equal, I want to
> write some tests which have no notion of "expected" and
> "actual." I would like

assert(navy_blue ===
> midnight_blue)

to pass--and I would like
> assert(grape === salmon) to fail with a
> message such as grape does not equal salmon.
>
> My math brain thinks of equality as symmetric. Therefore,
> I find an asymmetric failure message such as
> expected this but got that a bit disturbing.
>
That may be better. Michael didn't like "left 3 does not equal right 4", but "3 does not equal 4" should be pretty clear. The order in the message would be the order in the === expression. What would others think of that approach instead. The change would be:

assert(first === second)

And the message would be

<first> does not equal <second>

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 7:31 AM

assert(navy_blue ===
> midnight_blue)

to pass--and I would like
> assert(grape === salmon) to fail with a
> message such as grape does not equal salmon.

Except we usually have an expression rather than a literal, so we'd have something like assert(foodType() == salmon) and then we end up looking at "grapes != salmon" and say "Where did 'grapes' come from?" If the test output isn't right next to your code, you have to hunt back to the code to figure out what's up.

> My math brain thinks of equality as symmetric. Therefore,
> I find an asymmetric failure message such as
> expected this but got that a bit disturbing.

Yes, and frankly, I think that the assertEquals nomenclature from JUnit was a mistake. Equality isn't what the assertion is about, it's about testing an actual given an expectation and it should've been named that way, with a name that indicated its asymmetry.

It's easy to say that all of these stuff doesn't matter, but it does. Good affordances eliminate a lot of confusion. If you can eliminate confusion with a good name, why not?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 8:03 AM

> Yes, and frankly, I think that the assertEquals
> nomenclature from JUnit was a mistake. Equality isn't
> what the assertion is about, it's about testing an actual
> given an expectation and it should've been named that way,
> with a name that indicated its asymmetry.
>
> It's easy to say that all of these stuff doesn't matter,
> but it does. Good affordances eliminate a lot of
> confusion. If you can eliminate confusion with a good
> name, why not?
>
In Scala, you can say assert(a == b) anywhere. "assert" is a method defined in PreDef, which is imported into every Scala file implicitly. My take was that === means essentially the same thing as ==, except with a more helpful message contained in the AssertionError that results if there's a failure. assert(a == b) and assert(a === b) differ only in the contents of that message. Otherwise their behavior is the same. So === does seem about equality to me conceptually.

I toyed with other names for ===. It is a bit long, and I found it easy to forget the extra = sign sometimes. But Haskell's @=? looks pretty ugly to me at first blush. ==? could work. But then you have to hit two different keys, = and ?, and the shift key for the ?. People who use ScalaTest will type this a lot.

assert(a ==? b)

It looks kind of funky to me.

Elizabeth Wiethoff

Posts: 89
Nickname: ewiethoff
Registered: Mar, 2005

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 8:03 AM

I was thinking of grape and salmon as colors. :-)

> Yes, and frankly, I think that the assertEquals
> nomenclature from JUnit was a mistake. Equality isn't
> what the assertion is about, it's about testing an actual
> given an expectation and it should've been named that way,
> with a name that indicated its asymmetry.

Nevertheless, JUnit's assertEquals cannot discern that its right argument comes from a method under test and its left argument is the expected output of that method. We're accustomed to using it that way because the JavaDocs say assertEquals(expected, actual) and because consultants and book authors say that's what assertEquals is for. ;-) Even Dive into Python uses it like that.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 8:10 AM

> In Scala, you can say assert(a == b) anywhere. "assert" is
> a method defined in PreDef, which is imported into every
> Scala file implicitly. My take was that === means
> essentially the same thing as ==, except with a more
> helpful message contained in the AssertionError that
> results if there's a failure. assert(a == b) and assert(a
> === b) differ only in the contents of that message.
> Otherwise their behavior is the same. So === does seem
> about equality to me conceptually.
>
By the way, I didn't have to put === inside an assert. It could easily have been done this way, leaving off the assert:

a === b

Just stick that in your code and it would do the assertEquals(a, b)-like comparison. Even though it was a more finger typing, though, I thought require it to be wrapped in "assert" looked better.

assert(a === b)

So another option would be something like

a ==? b

and no assert. But I still think I prefer assert(expected === actual), using the JUnit order.

Elizabeth Wiethoff

Posts: 89
Nickname: ewiethoff
Registered: Mar, 2005

Re: ScalaTest 0.9 Released

Posted: Jan 16, 2008 8:19 AM

Well... not everything has to smell like Java and a testing framework doesn't have to smell like JUnit. So the question is, what nomenclature/syntax is in the spirit of Scala?

Flat View: This topic has 58 replies on 4 pages [ « | 1 2 3 4 | » ]

Previous Topic

Next Topic