Weblogs Forum - Are Tests First Class Clients?

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Are Tests First Class Clients?

57 replies on 4 pages. Most recent reply: Aug 2, 2005 9:23 AM by Jim Cakalic

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 57 replies on 4 pages [ « | 1 2 3 4 | » ]

unmesh

Posts: 15
Nickname: unmesh
Registered: May, 2003

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 6:35 AM

I guess tests are more than clients for the classes. They are more like invariants for the object. In that sense they complement the type system of any language. So they should know everything about the object, including the private fields/methods. In Java only way you can give access to these fields is by making them package scoped or public. This pollutes the API, but this is the only way and its Ok when it serves the larger good.
Polluting the API like this is a larger problem and is not only in the case of writing tests. There are some situations where we need to make a system more modular and have to compromise encapsulation. Take for example drawing different kind of shapes on different kind of surfaces. We have to use techniques like double dispatch or visitor. The class which implements the drawing routine (the visitor) may need access to all the private fields of different Shape objects. Only way to give such access in Java is to add a public accessor. So it will pollute the API for other clients.
In C++ you can define friend function/field for this purpose.
In order to solve the problem you discuss, we need something like friend functions in Java

Rick Kitts

Posts: 48
Nickname: rkitts
Registered: Jan, 2003

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 7:06 AM

> Option 1 is possible to do. The programatic contract of
> merge method is simple and easy to test, just
> do the merge. But using the output of merge
> to test the controllers is a different story. That's a lot
> more hairy, because what comes out of that merge is really
> user interface. The controller output data is very buried
> inside of HTML and JavaScript other web stuff that changes
> often. It would be a lot of work to effectively screen
> scrape the data out of the HTML, and every time you
> updated the look and feel of your site, you'll risk
> breaking the tests of the controllers, even though it was
> the screen scraping that broke, not the controllers.

Why is a usage of the class coupled to the testing of the class? I think the contract of Merger exists outside the context of the specific set of files used to define your UI. So some tests might be:


public void testCorrectlySubstitutesContextValueIntoTemplate()

public void testNoValueInContextDoesntChangeTemplate()

public void testToStringReturnsSameValueAsWhenUsingAWriter()

public void testThrowsWhenConstructedWithNullTemplateName()

public void testThrowsWhenConstructedWithNullContext()

public void testThrowsWhenConstructedWithNullResourceBundle()

This might seem like a lot for such a simple class, but this appears to be the contract (more or less). I don't know enough about Velocity but personally I would try to write the class so it didn't use Strings that name Files, but a Reader or IOStream. That way the tests are self contained. I'd then come up with something like:



public class MergerContext{
   public MergerContext(String templateName, Map context);

   public Reader /* or InputStream */ getTemplateData();

   public Map getContext();
}

modify Merger to have a single constructor...



public Merger(MergerContext mergerContext)

And use a mockish thing in tests



public class MergerContextMock extends MergerContext{
    // Override and this would use a StringReader  or whatever
    public Reader /* or InputStream */ getTemplateData();
}

> I was in China at the time, so I was upside down and that
> may have affected my perspective. But I got to wondering
> why I always think of tests as being second class citizens
> of my application that don't deserve any special support
> in the API.

I think there is a difference between polluting interfaces and designing for tests. As my example above shows I've added an entirely new class simply to make tests easier. However I feel that the design would stand up and nobody would consider it anything more than verbose. Which is to say if you didn't know that my design center was testing, you might look at the above and say it was a little OO overboard, but I doubt anyone would call the classes polluted.

> I want to have very good test coverage for
> this new architecture effort. The tests will always be
> there alongside the application. They are a very important
> part of it, so why should I feel bad about treating them
> like I treat other clients?

They're not exactly like other clients. I think they deserve changes in class structure and design. I do not think they deserve methods all of their own. This might sound backwards but I think after writing lots of tests you would come to the same conclusion.

Antonio R. Rodríguez S.

Posts: 6
Nickname: rodant
Registered: Jan, 2005

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 7:11 AM

Hello, I have made this question to my self several times and thas my opinion.

I also have a problem changing an API for test purposes, if I can't imagine any use for the "normal clients". I consider those situation as hints for design problems (maybe in the tests) and try to solve it in this way.

It is difficult to evaluate your concrete problem without the sources. As far as I can understand, you are testing if the controller is returning right values. I would use a Mock Merger instead of the right one and the verify methods will move to the Mock Merger. So you can keep your Merger untouched.

Best regards,
Antonio Rodríguez S.

Demetrie Gerodimos

Posts: 2
Nickname: amilla
Registered: Jan, 2005

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 9:20 AM

From an engineering point of view, this is OK - if not necessary - if you embrase DFT (design for testability).

For example, there are a number of transistors left on a chip that are only probed in the factory right before the die is cut, and current never runs through them again. We are bound to get more use from our test interfaces.

Designing for manufacturing, transport, testability, has realy useful parallels in software engineering.

J. B. Rainsberger

Posts: 12
Nickname: jbrains
Registered: Jan, 2004

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 9:40 AM

The verify() methods are equivalent to get methods, in that they expose data for comparison, so I recommend going one step further: implement equals() to compare Merger instances by their current state. This way, your tests can simply use assertEquals() to compare expected state with actual state, without exposing the internals of that state.

This might seem strange, since the class name is Merger. A verb-named class implementing equals()? Well, I don't like verb-named classes, because those are generally procedures, rather than objects. If procedural code is appropriate here, then I'd expect a design like this:

MergeResult result = merger.merge(templateName, velocityContext);

Here, result contains the string-based output along with any metadata describing the rendered page. (If there is none, then maybe MergeResult is just String.)

To test whether the controller passes the expected parameters to the merger, give the controller a mock merger that expects merge() to be invoked with a certain template name and Velocity context. This way you can verify what you wanted to verify without looking inside an object for its private data. You verify the parameters passed into the object, and assume the object knows what to do with them.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 10:39 AM

> The verify() methods are equivalent to
> get methods, in that they expose data for
> comparison, so I recommend going one step further:
> implement equals() to compare
> Merger instances by their current state. This
> way, your tests can simply use assertEquals()
> to compare expected state with actual state, without
> exposing the internals of that state.
>
Actually, the verify methods don't expose data for comparison. They use the private data and either return quietly or throw a TestFailedException.

I considered the equals approach you suggest, and it is a good suggestion. But the reason I didn't go that way is that I wanted more information passed to TestFailedException than just the contents of the Mergers are different. I want to find out how they are different.

> This might seem strange, since the class name is
> Merger. A verb-named class implementing
> equals()? Well, I don't like verb-named
> classes, because those are generally procedures, rather
> than objects. If procedural code is appropriate here, then
> I'd expect a design like this:
>

MergeResult result = merger.merge(templateName,
> velocityContext);

> Here, result contains the string-based output
> along with any metadata describing the rendered page. (If
> there is none, then maybe MergeResult is just
> String.)
>
Actually, "Merger" is a noun. It usually means two companies companies joining forces, so it doesn't quite fit here, but it is catchy and distinctive and seems to serve us well in discussions. The verb "merge" is the name of one of its methods.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 10:48 AM

This morning over bagels I came up with an idea of how to solve this thorny issue. I often find myself making my designs slightly worse for the sake of making them more easily testable by making private methods package access. Java does allow my test code to call those private methods, but it requires using reflection. This approach requires a lot of code, and because I'm using reflection, I break the tests when I change the name of the private method or move it, even though I'm using a refactoring IDE. Of these two options, I have always chosen to just make the private methods package access.

But in the case of Merger, it occurred to me this morning that it might be worth it to use reflection. I could move the verify methods back to the test area, but instead of adding get methods to Merger, just write some "get" methods in the test code that uses reflection to grab the private data out of Merger.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 11:13 AM

> They're not exactly like other clients. I think they
> deserve changes in class structure and design. I do not
> think they deserve methods all of their own. This might
> sound backwards but I think after writing lots of tests
> you would come to the same conclusion.

Well, maybe. I see your point, but if someone came down from outer space and we tried to explain this to them, I suspect they'd think we were being irrational.

Except for my last post about using reflection, in every other way we've talked about doing this, the presence of testing makes the API design worse in some way. Testing is a use case of the API, and it influences the design. It is like there's some kind of Heisenberg Uncertainty Principle going on. You can't observe an API with tests without changing that API (my apologies to physicists). Given that's the case, why wouldn't we choose the option that has the minimum negative impact on the design?

Rick, you suggested creating a MergerContext class to pass to the Merger constructor. This class would be a holder for the data, like ModelAndView in Spring. The controllers could return a MergerContext, which the tests could test by calling the get methods. The way this makes the API worse is it adds another class, which adds surface area. And it is done for the sake of testing. Wouldn't it be better to just put the get methods into Merger itself? It too makes the design worse, but I think that would incur a lower usability cost.

J.B. Rainsberger and others suggested using mock objects. There are several ways to do this. I could make Merger into an interface with one real and one mock implementation. Or I could leave Merger as a class and make a mock subclass, giving the Merger subclasses access to the private data in some way. I then would need to define a MergerFactory that I would pass down to the controllers. They would use the factory to create their Mergers. Someone suggested I make Merger mutable, so I just pass one into the controllers. That doesn't work in our case because controllers return a tree of Mergers, and the controllers are responsible for knowing how many Mergers they'll need.

So the impact of the mock object approach to the API design is that it adds one factory class and transforms a single class Merger into either one class and a subclass, or one interface and two subclasses. How is this lower impact than adding two get methods or two verify methods to a single class merger? That person from outer space would be scratching its heads.

Rick Kitts

Posts: 48
Nickname: rkitts
Registered: Jan, 2003

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 12:27 PM

> So the impact of the mock object approach to the API
> design is that it adds one factory class and transforms a
> single class Merger into either one class and
> a subclass, or one interface and two subclasses. How is
> this lower impact than adding two get methods or two
> verify methods to a single class merger?

Hmm. So I've not examined my predjudices here in any depth and that will probably show. However, I think it is a different thing to have a testable design and a testable implementation. That is, it is the design which sucumbs to test pressures and not the implementation. The question seems to be, why is one better than the other? Hopefully that isn't entirely opaque as a statement.

I'm not sure, but I suspect this might come down to something like why getters are better than publics. 99% of the time you could get away with the public but the operative phrase is "get away". That's not any sort of an answer of course. I'll try to think about this in the next few days.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 12:56 PM

> Hmm. So I've not examined my predjudices here in any depth
> and that will probably show. However, I think it is a
> different thing to have a testable design and a testable
> implementation. That is, it is the design which sucumbs to
> test pressures and not the implementation. The question
> seems to be, why is one better than the other? Hopefully
> that isn't entirely opaque as a statement.
>
I would say that most people here seem to have the same attitude, and I have it too to a great extent. I imagine that one justification of the mock objects approach could be that it is a more "flexible" design in that in the future I could have a third implementation of the Merger interface. But I could also change the existing monolithic Merger to such a design when the time comes that I actually need multiple implementations. So I don't really buy that argument. But it may *feel* better to us for that reason.

The way I would describe my own philosophy is that I'd like to minimize the impact of the testing use case to my APIs, and whenever possible, make it zero. That helps usability by keeping the testing and tested concerns nicely separated. I'm hoping the reflection approach I described earlier will help me make this one have zero impact.

Andrew Thompson

Posts: 2
Nickname: lordpixel
Registered: Jul, 2004

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 4:03 PM

Is the solution to the extra getters not simply classic inversion of control?

ie, starting from

public class Merger {
 
  String getFoo() { ... } //package access
}
 
public class MergerTest {
  public void testFoo() {
    Merger m = new Merger();
    String foo = m.getFoo();
... etc
  }  
...
}

Then one solution is:

public class Merger {
    private String foo = ...
 
    void acceptTest(MergerTest test) {
        test.setFoo(foo);
    }
}
 
public class MergerTest {
  private String foo;
 
  public void setUp() {
    Merger m = new Merger();
    m.acceptTest(this);
  }     
 
  public void setFoo(String foo) {
    this.foo = foo;
  }
 
  public void testFoo() {
    ... do something with foo
  }  
...
}

Many variations are possible, clearly for complex tests you might need multiple accept methods that would setup different values in the test case. One could imagine Merger having to call multiple steps on the test case, or vice versa.

However, the key points are:

* the acceptTest method is package level and thus won't pollute JavaDoc
* it requires an argument of type MergerTest, which makes it very hard to call accidently

This makes it very clear in the source code that this method is only used for testing. It also means that the Merger class remains in control - it decides what information to give to the test, not the other way around.

A malicious programmer could make their own class called MergerTest, so if the value of 'foo' is Top Secret then this is inappropriate (but then, they could just use a debugger to look at it).

Seems like it covers a lot of the objectsions. Its certainly very clear what's going on when you read Merger

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 4:58 PM

> Except for my last post about using reflection, in every
> other way we've talked about doing this, the presence of
> testing makes the API design worse in some way. Testing is
> a use case of the API, and it influences the design...
>
Time for a philosophical question here. (I'm good at those, especially as I haven't been following the code closely.)

My philosophical question is this:
If the test needs to examine internal state, then
hasn't it broken encapsulation? The test then becomes
a dependent application that breaks when and if the
implementation changes.

Of course, that's probably the point you're making. But the other half of the question is, why isn't there anything that's testable without looking at internal state?

Now, I also know that it's not always possible to preserve beautiful philosophical ideals like encapsulation. But why is that, I wonder? Is there no way to verify correct behavior without examining internal state? Or is granularity the issue--the fact that by the time you get to a verifiable output, so much code has come and gone that the "unit" test covers too much ground?

From the testing perspective, the answer in that case is that the code *should* have more APIs/data values exposed in order to make testing possible. From a design perspective, that also breaks up a monolothic conglomeration into byte-sized chunks that can be recombined in new and interesting ways.

That's all very general and philosophical, I know. I'm looking forward understanding why this particular code is an exception (if it is) to the general idea that stuff should be encapsulated, and fully encapsulated code still has testable outputs.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 10:22 PM

> My philosophical question is this:
> If the test needs to examine internal state, then
> hasn't it broken encapsulation? The test then becomes
> a dependent application that breaks when and if the
> implementation changes.
>
Yes.

> Of course, that's probably the point you're making. But
> the other half of the question is, why isn't there
> anything that's testable without looking at
> internal state?
>
I would say that anything that needs to be tested is by definition testable from the public interface. If some behavior isn't visible from the public interface, then philosophically speaking, I don't see why it needs to be tested. Running such code would be like a tree falling in a forest with no one around to hear it. It probably makes a sound, but it is really irrelevant because no one can observe it.

The reason I occasionally want to write tests that break encapsulation, such as testing private methods or checking private data, is because it is too expensive to do the test through the public interface.

Merger is an example. I could test controllers by looking at the output of merge, but that would require screen scraping. Such tests would be very time consuming to write and fragile once written. They would require that we spend a lot of time writing the tests and fixing them every time we made a look and feel change to the website. I have made the judgement that in this case the cost of testing Merger through the public interface is higher than the cost of breaking encapsulation by "polluting" the API and testing it that way.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 10:46 PM

> Is the solution to the extra getters not simply classic
> inversion of control?
>
I believe this moves, rather than sovles, the problem. Instead of extra getters we have an extra acceptTest method and a new type, MergerTest, that client programmers have to wrap their heads around. I'd say that from an encapsulation standpoint your suggestion is a better choice than the getters, but not from a simplicy standpoint. And I'd judge the verify method approach I actually implemented is as good on encapsulation and better on simplicity than the acceptTest approach.

> * the acceptTest method is package level and thus won't
> pollute JavaDoc

Unfortunately, in this particular case the acceptTest methods would in practice need to be public, just like the verify methods. The reason is that these methods are needed when testing controllers, and controllers are in a different package than Merger.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Are Tests First Class Clients?

Posted: Jan 27, 2005 11:56 PM

Introducing Aspects into the code is like using a sledgehammer to crack a nut, with the added bonus that you'll end up with non-standard java code that no-one else can maintain.

Flat View: This topic has 57 replies on 4 pages [ « | 1 2 3 4 | » ]

Previous Topic

Next Topic