Weblogs Forum - What are Your JUnit Pain Points, Really?

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
What are Your JUnit Pain Points, Really?

65 replies on 5 pages. Most recent reply: Nov 13, 2009 2:01 PM by Nemanja Trifunovic

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 65 replies on 5 pages [ « | 1 2 3 4 5 | » ]

Tasos Zervos

Posts: 17
Nickname: tzervos
Registered: May, 2003

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 4, 2009 10:19 AM

Reply

Advertisement

+1

Also, if you think in terms of black-box and glass-box unit testing, you are much better off with the former. With little dependending on your implementation you get very easy refactoring.

Another way to think of TDD is: the tests you create form your API's first client (and therefore influences you towards keeping unit-test code at the right level).

In general the more decoupled is your design the better unit-testing experience you get in relation to future refactorings. And this is where TDD helps because it encourages decoupling. (That is, if you're willing anyway...)

Bob Lauer

Posts: 5
Nickname: rfl4711
Registered: Jul, 2008

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 5, 2009 9:57 AM

Reply

> My biggest JUnit pain point is that once I've invested
> into writing lots of unit tests, I can't do major
> refactorings of the codebase without throwing away all
> that investment. This is probably not a JUnit only pain
> point.

Maybe we have to evolve programming beyond TDD:


Level 0: Get it to work
Level 1: Assert that some parts behave in a certain way
 (unit testing)
Level 2: Assert that something did not work in a certain
way before you changed it toward a certain requirement
 (TDD)
Level 3: Assert that every requirement can be deconstructed again by specifying what functionality implements it.

Anyone willing to go beyond level 2?

Robert C. Martin

Posts: 111
Nickname: unclebob
Registered: Apr, 2003

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 5, 2009 10:19 AM

Reply

> If you obsessively do TDD, you write tests for code that
> you are pretty much guaranteed to throw away.

Dingoes Kidneys! I've been writing FitNesse since 2001, and I've been pretty obsessive about TDD. I have 61,000 lines of Java code, 27,000 of which are in the 2100 unit tests. I do not have this problem.

> And when you
> do that, you will have to refactor your tests or rewrite
> them completely.

Mastodon Tonsils! I have added over 500 tests to FitNesse in the last year, and have refactored mightily. The impact upon the existing tests has been minimal.

Now, maybe I just write better tests than others. Here's a hint. I write them first.

Robert C. Martin

Posts: 111
Nickname: unclebob
Registered: Apr, 2003

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 5, 2009 10:28 AM

Reply

> My biggest JUnit pain point is that once I've invested
> into writing lots of unit tests, I can't do major
> refactorings of the codebase without throwing away all
> that investment.

1. Refactor ruthlessly at all times. The more you refactor at the beginning, the more insulated your tests become.

2. Extract, extract, extract. Extract till you drop. Keep on shrinking your methods until they are very small. You want a few public methods and lots and lots of little private methods. But you only test the public methods!

3. MOCK! Break the dependencies between different parts of your system by writing appropriate mock objects (nowadays called "Test Doubles"). Avoid using a mocking tool for this unless absolutely necessary. Write the mocks by hand, and use interfaces to insulate them from the rest of the system.

4. Take care of your tests the way you take care of your code. REFACTOR YOUR TESTS! Keep them clean. Keep them well organized. Invest some design time into them.

Jess Holle

Posts: 20
Nickname: jessh
Registered: Jan, 2009

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 5, 2009 12:38 PM

Reply

> 3. MOCK! Break the dependencies between different parts of
> your system by writing appropriate mock objects (nowadays
> called "Test Doubles"). Avoid using a mocking tool for
> this unless absolutely necessary. Write the mocks by
> hand, and use interfaces to insulate them from the rest of
> the system.

I disagree. Mocking is a last resort. When you mock you are asserting implicit assumptions about other subsystems. The real question is what really happens when you put the systems together -- not how they behave with your mocks. Hand mocking also places onerous requirements on APIs forcing them to break encapsulation where there is no other reason/requirement for this. A tool like JMockit avoids this.

Daniel Sobral

Posts: 80
Nickname: dcsobral
Registered: Aug, 2008

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 5, 2009 4:55 PM

Reply

> > 3. MOCK! Break the dependencies between different parts
> of
> > your system by writing appropriate mock objects
> (nowadays
> > called "Test Doubles"). Avoid using a mocking tool for
> > this unless absolutely necessary. Write the mocks by
> > hand, and use interfaces to insulate them from the rest
> of
> > the system.
>
> I disagree. Mocking is a last resort. When you mock you
> are asserting implicit assumptions about other subsystems.
> The real question is what really happens when you put the
> e systems together -- not how they behave with your mocks.
> Hand mocking also places onerous requirements on APIs
> s forcing them to break encapsulation where there is no
> other reason/requirement for this. A tool like JMockit
> avoids this.

I have no clue why do you think that. You mock if object A depends on (ie, call methods on) object B. Where is encapsulation broken? Where are the onerous requirements?

As for implicit assumptions, whenever two objects interact you'd better have explicit statements about what the caller can expect from the callee. If you can't make such explicit statements, then the two objects aren't decoupled and, in fact, are violating encapsulation.

One good testing technique, in fact, states that, when mocking B in a test for A, for every method M with parameters P called on the mocked B, resulting in R, you should have a test on B to check if method M with parameters P will truly return result R.

Now, when you DON'T mock, not only you may be hiding the design problems mentioned, but you are mixing the behaviors of A and B in the same test. Not only that means a bug in one can hide a bug in the other, but, when a test fails, you don't know which has failed.

Of course, if the mocked object is little more complex than the mock itself, there's no point in mocking it.

Jess Holle

Posts: 20
Nickname: jessh
Registered: Jan, 2009

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 6, 2009 6:31 AM

Reply

> I have no clue why do you think that. You mock if object A
> depends on (ie, call methods on) object B. Where is
> encapsulation broken? Where are the onerous requirements?

Many mocking systems require the class being mocked expose interfaces, public methods or mutability that are otherwise quite inappropriate for the design.

Nemanja Trifunovic

Posts: 172
Nickname: ntrif
Registered: Jun, 2004

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 6, 2009 8:26 AM

Reply

Funny enough, I found "mocking" to be useful in development. In a couple of projects I worked on, we would come up with a mock (we called it "dummy") component that would hold the fort for the "real" one until is completed, and enable developers who depend on it to do at least some work instead of being stuck.

As for the tests, I don't mock anything anymore. My tests need to be run on the real thing to make me confident I didn't break anything. I want even the testing hardware to be as similar as production environment as possible (when applicable, of course).

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 6, 2009 9:01 AM

Reply

> Funny enough, I found "mocking" to be useful in
> development. In a couple of projects I worked on, we would
> come up with a mock (we called it "dummy") component that
> would hold the fort for the "real" one until is completed,
> and enable developers who depend on it to do at least some
> work instead of being stuck.

Yes, this can be useful but make sure that the mocked methods throw an exception (or fail loudly) or you might waste hours trying to find why your code isn't working because it's invoking empty methods...

--
Cedric

Nemanja Trifunovic

Posts: 172
Nickname: ntrif
Registered: Jun, 2004

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 6, 2009 9:11 AM

Reply

>
> Yes, this can be useful but make sure that the mocked
> methods throw an exception (or fail loudly) or you might
> waste hours trying to find why your code isn't working
> because it's invoking empty methods...
>
> --
> Cedric

I would not mix "mocked" code with the production one. What we did was create a complete "dummy" component, work with it until the real one is done and then just replace everything. The "dummy" is not supposed to complain because that would defeat its purpose - it just pretends it is doing the real thing and returns results that look like real ones. For instance, imagine a data layer component that never connects to a database but stores data in memory, or even does not store it at all - just returns "dummy data" when asked.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 6, 2009 11:59 AM

Reply

> As for the tests, I don't mock anything anymore. My tests
> need to be run on the real thing to make me confident I
> didn't break anything. I want even the testing hardware to
> be as similar as production environment as possible (when
> applicable, of course).

The potential value I see in mocking is for regression testing. I can build scripts of functional test cases that I run whenever I want to validate that my last change didn't affect something I didn't mean it to.

This works great for situations where part of the script is creating the data that will then be validated. But a lot of the things I would like to test depend on data that is in constant flux.

I see two possibilities, I either set up tables only for testing these types of things and never update them or I mock the data such that it is consistent.

End-to-end testing isn't going to go away. What I want to do with this is check that my code still produces the same results given the same data. I can then isolate problems caused by code changes from problems external to the code.

Al Scherer

Posts: 3
Nickname: ascherer
Registered: Nov, 2009

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 9, 2009 7:58 AM

Reply

Hi,

You mentioned... "But you only test the public methods!"

Why wouldn't we want to test our private methods, too?

Thanks,
Al

Daniel Sobral

Posts: 80
Nickname: dcsobral
Registered: Aug, 2008

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 9, 2009 8:35 AM

Reply

> Hi,
>
> You mentioned... "But you only test the public methods!"
>
> Why wouldn't we want to test our private methods, too?

It wasn't me mentioning it, but... let's recap TDD.

Test Driven Development means you write a test, and then write the code to make the test work. This way, the development you do is driven by the tests you write.

So, let's go back to those tests you write. What tests should you do? First, you don't think of implementation. It should be no concern whatsoever to you what the implementation is when you write the tests.

What you should concern yourself with is... what do you use that class for? Your tests are effectively a double for the client code of that class.

This is one important point, for many reasons. On of those reasons is that you don't make your class more complex than strictly needed to handle the requirements of the client code, as represented by the tests.

So, the direct consequence of it is that you write tests for the API of the class, and the contract that class has to abide by.

And those things are represented by the public methods. The private methods are just implementation, and you do not concern yourself with it. You just ensure that the API is providing what it must.

Which goes back to many of Uncle Bob's points. For instance, mock a lot -- if you test an API, then:

1) Having an interface for it is not only reasonable, but a good practice.

2) Having an stable API is important. Control its version as you expect the libraries you use to control theirs: minor revisions when the API changes in ways that break forward compatibility, and major revisions when the API changes in ways that breaks backward compatibility. Yes, for pretty much every class you write.

Any changes to *how* you go about doing the stuff becomes invisible. It reduces bugs introduced by subtle changes in a class that you think shouldn't be a problem, and makes you work that little bit harder at maintaining the API stable.

Also, because you write the API from the PoV of the client code, you are already optimizing API for usage, which is the main reason for refactoring that changes the public methods of a class.

Or that, at least, is the theory. And it is a learning experience all of its own.

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 9, 2009 8:45 AM

Reply

> Hi,
>
> You mentioned... "But you only test the public methods!"
>
> Why wouldn't we want to test our private methods, too?

It's still a debated issue, a lot of people think that testing private methods is useful.

For example, if I write a class that has one public method and ten private methods, that public method is probably fairly complex (it has a lot of different code paths) and testing it will be delicate.

In this case, I don't see any harm in testing the smaller private methods too. It also reduces the number of tests to case dramatically.

For example, consider:

if (a && b) {
f();
} else if (c || b) {
g();
} else if (d) {
h();
}

That's a lot of code paths (four booleans, potentially sixteen tests) but if you just test the private methods, the number of tests comes down to 3 + testing your public method.

A few more thoughts on this topic:

http://beust.com/weblog/archives/000303.html

--
Cedric

John Zabroski

Posts: 272
Nickname: zbo
Registered: Jan, 2007

Re: What are Your JUnit Pain Points, Really?

Posted: Nov 9, 2009 8:50 AM

Reply

> What I want to do with this is check that my code still produces the same results given the same data.

...and simulatenously be tolerant to changes in syntax, semantics and pragmatics.

Especially for a Facade like SQL.

Flat View: This topic has 65 replies on 5 pages [ « | 1 2 3 4 5 | » ]

Previous Topic

Next Topic

Sponsored Links

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use