SuiteRunner Forum - tearDownFixture not executed when a test fails. Bug or "feature"?

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

SuiteRunner Forum
tearDownFixture not executed when a test fails. Bug or "feature"?

2 replies on 1 page. Most recent reply: Feb 28, 2003 9:01 AM by Andrew Matthews

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 2 replies on 1 page

Andrew Matthews

Posts: 9
Nickname: taxman
Registered: Feb, 2003

tearDownFixture not executed when a test fails. Bug or "feature"?

Posted: Feb 27, 2003 12:36 PM

I have recently made the discovery that a Suite's TearDownFixture method is not called after a test failure, or when a test throws an Exception of any kind. I examined the source code for Suite.executeTestMethod and discovered this:

try {
if (invoker == null) {
method.invoke(this, args);
} else {
invoker.invoke(this, method, args);
}
}

The culprit is in the two catch blocks that follow (which I have not included for brevity. Both of them contain a return statement that doesn't allow the tearDownFixture method to execute.

Is this intentional, or is it a bug? If it is intentional, it should be prominently documented in Suite's documentation.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: tearDownFixture not executed when a test fails. Bug or "feature"?

Posted: Feb 27, 2003 2:00 PM

Thanks for pointing this out. I did intentionally implement the behavior you correctly describe, but now it seems wrong to me.

Back when we were putting this together, I looked at the bug reports and enhancement requests for JUnit, and one request complained that in JUnit, the test method was called even if the setUp method failed with an exception. I felt that was the wrong behavior, so I made sure executeTestMethods didn't do that. If setUpFixture fails with an exception, neither the test method nor tearDownFixture is called.

But then I went ahead and did a similar thing if the test method fails. If the test method fails with an exception, I don't call tearDownFixture. Looking at it now I think this is clearly wrong. tearDownFixture should most likely be executed if setUpFixture succeeds, because they work in tandem. setUpFixture may be claiming resources that need to be released in tearDownFixture.

So, unless someone has an objection, I'll change that in the next release. Unless setUpFixture fails with an exception, I'll make sure executeTestMethods invokes tearDownFixture, regardless of whether the test method throws an exception or returns normally.

And, I'll document the behavior in contract for executeTestMethods. You also discovered that the JavaDoc comment for executeTestMethods is too vague. I wrote it back when we didn't have setUpFixture and tearDownFixture methods. We were trying to see if we could get rid of them, but ultimately decided they were useful to a lot of people. So we added them, but I forgot to update the JavaDoc comment.

Anyway, invoking tearDownFixture if setUpFixture succeeds seems like the appropriate default behavior for executeTestMethods. It is what I expect most people woudl expect and desire. It may not be what everyone wants all the time, but in such cases those people can override executeTestMethods and implement different behavior.

I am hoping to have the next release on or slightly before March 10. I am planning to add support for user-specified properties in the Recipe file for the next release, so there will be an enhancement as well as this executeTestMethods bug fix.

Thanks for reporting this bug. If anyone doesn't think this change is the right thing to do, please speak up.

Andrew Matthews

Posts: 9
Nickname: taxman
Registered: Feb, 2003

A quick and dirty workaround...

Posted: Feb 28, 2003 9:01 AM

In the meantime, to work around this problem, I wrap the meat-and-potatoes of my test code in a try-catch block and call tearDownFixture in a finally block. Seems to work quite nicely.

Flat View: This topic has 2 replies on 1 page

Previous Topic

Next Topic


	Web Artima.com