The Artima Developer Community
Sponsored Link

Weblogs Forum
Are Tests First Class Clients?

57 replies on 4 pages. Most recent reply: Aug 2, 2005 9:23 AM by Jim Cakalic

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 57 replies on 4 pages [ « | 1 2 3 4 | » ]
Jared MacDonald

Posts: 13
Nickname: jared
Registered: Oct, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 9:11 AM
Reply to this message Reply
Advertisement
> I can test a controller through Merger's
> public interface, then, by passing a
> StringWriter into the merge
> method, and then just comparing the outputted string with
> a correct output. The trouble is that it is a pain to do
> this, because the template changes all the time.

But can't you set up Merger with a very small, testable template?

It sounds to me like the verify() methods are helpful, but what ultimately matters is the HTML that's rendered onto the page -- and that is what you get back from the public API. The verify() methods won't do you any good if the actual output of merge() is wrong. So I'd strongly be in favor of somehow testing that merge() method, and indeed, actually verifying the HTML contents, employing some sort of mock template so that the result isn't 100s of lines long.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 9:33 AM
Reply to this message Reply
Hi Jared,

> But can't you set up Merger with a very small, testable
> template?
>

> It sounds to me like the verify() methods are helpful, but
> what ultimately matters is the HTML that's rendered onto
> the page -- and that is what you get back from the public
> API. The verify() methods won't do you any good if the
> actual output of merge() is wrong. So I'd strongly be in
> favor of somehow testing that merge() method, and indeed,
> actually verifying the HTML contents, employing some sort
> of mock template so that the result isn't 100s of lines
> long.

That's a good suggestion to test merge itself. We do that with a simple template, just to make sure the process of merging works.

But one of the whole reasons I wanted to have a clean MVC architecture was to facilitate testing. It is a pain to test views because they have a high rate of change. With a clean separation of controller and view, I can write unit tests that test the output of the controllers.

We have about a dozen controllers now, and will have at least a few hundred later. What comes out of our controllers is essentially a tree of Mergers, so that's the thing I'd like to use to test all those controllers. But testing them via Merger forces me to alter the interface of Merger in some way, either by adding get methods or by adding those verify methods.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 9:34 AM
Reply to this message Reply
Here's the code:

package com.artima.nextgen.webmvc;
 
import org.apache.velocity.context.Context;
import org.apache.velocity.Template;
import org.apache.velocity.VelocityContext;
import org.apache.velocity.app.Velocity;
import org.apache.velocity.runtime.RuntimeSingleton;
import org.suiterunner.Suite;
import org.suiterunner.TestFailedException;
 
import java.io.StringWriter;
import java.io.Writer;
import java.util.*;
 
import com.artima.website.ArtimaConstants;
 
/**
 * Contains a template name and context map, with which merges can be executed.
 * The merges can be executed in two ways: the <code>merge</code> and
 * <code>toString</code> methods. The <code>merge</code> method writes
 * the merge output to a passed <code>Writer</code>. The <code>toString</code>
 * method writes returns the output of the merge as a <code>String</code>.
 * <p/>
 * <p/>
 * This class can be placed inside a context map passed to a different Merger,
 * because its <code>toString</code> method does a merge and returns the result
 * as a <code>String</code>. This allows controllers to build documents as trees
 * of <code>Merger</code>s.
 */
public class Merger {
 
    // I pass a Map to the constructors, not a Velocity context, because I want
    // to do a defensive copy of the contents of the map. Since I am
    // going to do a defensive copy anyway, there's no need to have
    // controllers married to the velocity Context interface at all.
    // The reason I want to do a defensive copy is because people could
    // add in to the context passed in a copy of this Merger, which would
    // result in a cycle in the tree. That would cause the JVM to run out
    // of stack space when attempting a merge. Not a good thing. By doing it
    // this way, I don't even need to check for cycles here in the constructor,
    // because they are by definition impossible. You can't add this Merger to
    // the contextMap passed to this Merger's constructor before this Merger's
    // constructor has completed. Such a feat would require time travel. - bv 11/9/2004
    //
 
    private String templateName;
    private Context context;
 
    /**
     * Construct a new <code>Merger</code>. The passed contest map
     * may contain other <code>Merger</code>s.
     *
     * @param templateName the name of the template
     * @param contextMap   a context map
     * @throws NullPointerException if passed <code>templateName</code> or <code>contextMap</code>
     *                              is <code>null</code>
     */
    public Merger(String templateName, Map contextMap) {
 
        initialize(templateName, contextMap);
    }
 
    private void initialize(String templateName, Map contextMap) {
 
        if (templateName == null || contextMap == null) {
            throw new NullPointerException();
        }
 
        this.templateName = templateName;
 
        context = new VelocityContext();
 
        for (Iterator it = contextMap.keySet().iterator(); it.hasNext();) {
 
            String key = (String) it.next();
            Object value = contextMap.get(key);
 
            context.put(key, value);
        }
    }
 
    /**
     * Construct a new <code>Merger</code> with passed resource bundle and context
     * map. The resource bundle must contain the template name under the key
     * <code>"templateName"</code>. The passed contest map
     * may contain other <code>Merger</code>s.
     *
     * @param bundle     a <code>ResourceBundle</code> containing the name of the
     *                   template under the key "templateName".
     * @param contextMap a context map
     * @throws NullPointerException     if passed <code>bundle</code> or <code>contextMap</code>
     *                                  is <code>null</code>.
     * @throws java.util.MissingResourceException
     *                                  if passed <code>bundle</code> does not
     *                                  contain a value for the key <code>"templateName"</code>.
     * @throws IllegalArgumentException if the passed <code>bundle</code> contains a value
     *                                  for the
     */
    public Merger(ResourceBundle bundle, Map contextMap) {
 
        if (bundle == null || contextMap == null) {
            throw new NullPointerException();
        }
 
        Object o = bundle.getObject(ArtimaConstants.TEMPLATE_NAME);
        String templateName = null;
        try {
            templateName = (String) o;
        }
        catch (ClassCastException e) {
            IllegalArgumentException iae = new IllegalArgumentException("Value in bundle for templateName should be a String but is a: "
                    + o.getClass().getName());
            iae.initCause(e);
            throw iae;
        }
 
        initialize(templateName, contextMap);
    }
 
    /**
     * Merge the template and context contained in this <code>Merger</code>,
     * writing the output to the passed <code>Writer</code>.
     *
     * @param writer the Writer to which to write the merge output
     */
    void merge(Writer writer) {
 
        try {
 
            Template tmpl = RuntimeSingleton.getTemplate(templateName);
            tmpl.merge(context, writer);
        }
        catch (Exception e) {
            throw new RuntimeException("Missing template: " + templateName, e);
        }
    }
 
    /**
     * Merges the template and context contained in this <code>Merger</code>,
     * returning the result as a <code>String</code>. This
     * <code>toString()</code> implementation enables Merger's to be embedded in
     * Contexts.
     *
     * @return the result of the merge as a String
     */
    public String toString() {
 
        StringWriter writer = new StringWriter();
        merge(writer);
        return writer.toString();
    }
 
    public boolean equals(Object o) {
 
        boolean eq = false;
 
        if (o != null) {
 
            if (o.getClass() == Merger.class) {
 
                Merger m = (Merger) o;
 
                if (m.templateName.equals(templateName)) {
 
                    if (areContextsEqual(m.context, context)) {
 
                        eq = true;
                    }
                }
            }
        }
 
        return eq;
    }
 
    private static boolean areContextsEqual(Context a, Context b) {
 
        if (a == null || b == null) {
            throw new NullPointerException();
        }
        boolean eq = true;
 
        Object[] aKeys = a.getKeys();
        Object[] bKeys = b.getKeys();
 
        if (aKeys.length == bKeys.length) {
            for (int i = 0; i < aKeys.length; i++) {
                String aKey = (String) aKeys[i];
                if (b.containsKey(aKey)) {
                    if (!(a.get(aKey).equals(b.get(aKey)))) {
                        eq = false;
                        break;
                    }
                }
                else {
                    eq = false;
                    break;
                }
            }
        }
        else {
            eq = false;
        }
 
        return eq;
    }
 
    // For unit tests. Eventually, I'd like to make this private.
    public Map getContext() {
 
        Object[] keys = context.getKeys();
 
        Map map = new HashMap();
 
        for (int i = 0; i < keys.length; i++) {
 
            String key = (String) keys[i];
            map.put(key, context.get(key));
        }
 
        return map;
    }
 
    // For unit tests. Delete after making sure Frank didn't use this in the mean time.
    public String getTemplateName() {
        return templateName;
    }
 
    /**
     * Verify the context contained in this <code>Merger</code> is as expected. This method is
     * intended to be used with unit tests.
     *
     * @param expected the expected context map.
     * @throws org.suiterunner.TestFailedException
     *                              if the context map is not as expected
     * @throws NullPointerException if the passed <code>expected</code> <code>Map</code>
     *                              is <code>null</code>
     */
    public void verifyContext(Map expected) {
 
        if (expected == null) {
            throw new NullPointerException("Expected context Map cannot be null.");
        }
 
        verifyMapsEqual(expected, getContext());
    }
 
    /**
     * Verify the template name is as expected. This method is intended to be used with unit tests.
     *
     * @param expected the expected template name.
     * @throws org.suiterunner.TestFailedException
     *                              if the redirect URL is not as expected
     * @throws NullPointerException if the passed <code>expected</code> <code>String</code>
     *                              is <code>null</code>
     */
    public void verifyTemplateName(String expected) {
 
        if (expected == null) {
            throw new NullPointerException("Expected templateName cannot be null.");
        }
 
        Suite.verify(templateName.equals(expected), "templateName should have been: "
                + expected + ", but was: " + templateName);
    }
 
    private static void verifyMapsEqual(Map expected, Map actual) {
 
        if (expected == null || actual == null) {
            throw new NullPointerException();
        }
 
        Set expectedKeys = expected.keySet();
        Set actualKeys = actual.keySet();
 
        if (expectedKeys.equals(actualKeys)) {
            for (Iterator it = expectedKeys.iterator(); it.hasNext();) {
 
                String key = (String) it.next();
                Object expectedVal = expected.get(key);
                Object actualVal = actual.get(key);
 
                if (!expectedVal.getClass().isArray()) {
                    if (!expectedVal.equals(actualVal)) {
 
                        String msg = printMapsToString(expectedKeys, actualKeys, "Context key mismatch.");
                        throw new TestFailedException(msg);
                    }
                }
                else {
                    Object[] expectedArr = (Object[]) expectedVal;
                    Object[] actualArr = (Object[]) actualVal;
 
                    if (expectedArr.length == actualArr.length) {
 
                        for (int i = 0; i < expectedArr.length; i++) {
 
                            Object expectedEle = expectedArr[i];
                            Object actualEle = actualArr[i];
 
                            if (!expectedEle.equals(actualEle)) {
 
                                String msg = printMapsToString(expectedKeys, actualKeys, "Array elements not equal.");
                                throw new TestFailedException(msg);
                            }
                        }
                    }
                    else {
                        String msg = printMapsToString(expectedKeys, actualKeys, "Context key mismatch.");
                        throw new TestFailedException("Arrays are different lengths.");
                    }
                }
            }
        }
        else {
 
            String msg = printMapsToString(expectedKeys, actualKeys, "Context key mismatch.");
            throw new TestFailedException(msg);
        }
    }
 
    private static String printMapsToString(Set expectedKeys, Set actualKeys, String message) {
 
        StringBuffer buf = new StringBuffer();
        buf.append(message);
        buf.append(" ");
        buf.append("expectedKeys.size() is ");
        buf.append(expectedKeys.size());
        buf.append(", actualKeys.size() is ");
        buf.append(actualKeys.size());
 
        buf.append(". expectedKeys.toString() is ");
        buf.append(expectedKeys.toString());
        buf.append(". actualKeys.toString() is ");
        buf.append(actualKeys.toString());
        buf.append(".");
 
        return buf.toString();
    }
 
    String printToString(String msg) {
 
        if (msg == null) {
            throw new NullPointerException();
        }
 
        StringBuffer buf = new StringBuffer();
        buf.append(msg);
        buf.append(" ");
        buf.append(printMergerToString(this));
 
        return buf.toString();
    }
 
    private static String printMergerToString(Merger merger) {
 
        StringBuffer buf = new StringBuffer();
        buf.append("(");
        Map map = merger.getContext();
        for (Iterator it = map.keySet().iterator(); it.hasNext();) {
 
            Object key = (Object) it.next();
            Object val = map.get(key);
            buf.append(key.toString());
            buf.append(":");
 
            if (val instanceof Merger) {
                String s = printMergerToString((Merger) val);
                buf.append(s);
            }
            else {
 
                String s = val.toString();
                buf.append(s);
                buf.append(" ");
            }
        }
        buf.append(")");
 
        return buf.toString();
    }
 
 
}


Critique away...

Jean-Daniel Nicolet

Posts: 13
Nickname: jdnicolet
Registered: Oct, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 9:45 AM
Reply to this message Reply
Tests are definitely a first class client. Let's consider a higher level of granularity, that of a whole application, and not solely a single API (of a class or of a component).

At the application level, there are several roles that have different access privileges to the application. A way to expresse this otherwise is to say that different interfaces (e.g. API) are offered to different clients (e.g. roles).

In a complex deployment environment, where other people than developers control the production environment, we encounter the need to be able for people that do not know the business handled by the application to be able to assert that the application is nonetheless running properly, and this possibly in an automated way (i.e. with the help of say a watchdog application).

We come out with the need to require from the application designers to consider a special role in their application for the sole purpose of testing. Moreover, the test operation itself may need to access some confidential data in order to be sure this data access is reliable, but without letting the data leak out of the application (i.e. without displyaing it, for example, or without producing any visible output).

So clearly in this case, we require a "special API" for the sole purpose of testing.

If this need looks quite natural at the application level, the question is just to know if it is still reasonnable for a smaller component or even a single class, to offer different interfaces to different clients. The trick with the package access to particular methods is just a technical way to achieve the interface differentiation in a simple manner (i.e. without explicitly implementing access security).

In a language like C++, we would use a similar trick, albeit a little cleaner: the class under test would grant access to its internals to another, explicitly stated, test class. This mechanism is called "friendship declaration". It's a little cleaner in the sense that the granting class keeps control on who accesses it, whilst with the package access, you just restrict access to your "neighbourhood", but you don't know who exactly.

Rick Kitts

Posts: 48
Nickname: rkitts
Registered: Jan, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 10:20 AM
Reply to this message Reply
> Hi Rick,
>
[...snip...]
> I can test a controller through Merger's
> public interface, then, by passing a
> StringWriter into the merge
> method, and then just comparing the outputted string with
> a correct output. The trouble is that it is a pain to do
> this, because the template changes all the time.

I suspect you have not fully defined the contract of your interface...

[...snip...]
> So although I could test Mergers through the
> public interface, I find myself judging that the return on
> investment of doing that isn't worth the cost (in time
> spent doing it) compared to the return on investment of
> just testing the template name and context
> Map.

Ah. I've been here. I disagree more or less on a fundamental level. One value is in the learning. Put another way: If you cannot craft a test that do to input ranges then you probably have no contract worth speaking of. What I learned in writing tests is that while I might have thought an interface had a solid contract it didn't. And it didn't until I started writing tests.

> The latter approach also has a cost,
> which includes the usability cost of polluting the public
> interface of Merger with two
> verify methods. I made the judgement that I
> get a better return on investment using the latter
> approach.

As well as the loss of learning and the new world view that really, really, really making yourself write tests, and real tests that don't break APIs, brings.

I doubt if you would be half as skeptical as I was that spending more time writing tests than code was worthwhile.

William Tanksley

Posts: 1
Nickname: wtanksley
Registered: Sep, 2004

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 10:45 AM
Reply to this message Reply
Yes, tests are a first-class client and should be allowed to drive the interface. HOWEVER, just as with any client, tests should not be allowed to make the interface bad (incoherent, leaky, or error prone).

If you need information your current interface isn't giving you in order to tell whether your object works, you are being informed that your current interface isn't adequate.

I also believe that tests should be written in a way that makes them useful as "demonstrations" of typical, proper use.

Jan Ploski

Posts: 8
Nickname: jploski
Registered: Aug, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 11:03 AM
Reply to this message Reply
Since noone seems to have thrown it in yet, let me be the first: you could isolate the code which is only relevant in the test environment into an aspect, provided an aspect-oriented programming extension to Java which allows static weaving (such as AspectJ). Another technique reflecting the same school of thought is subject-oriented programming.

Even in the traditional OO world, it is common for a class to have multiple interfaces catering to different needs of different clients. If you let your clients use (Java) interfaces as a rule instead of public interfaces of concrete classes, you will find it easy to keep "dirty" things from the view, at the cost of introducing some code which appears redundant at first glance (namely, repeated lists of method signatures).

Alexander Jerusalem

Posts: 36
Nickname: ajeru
Registered: Mar, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 11:40 AM
Reply to this message Reply
I agree that the question of whether tests are first class clients is the crux. And in terms of influencing the requirements for the API, my answer is clearly no, they should _not_ and logically cannot be treated equally.

Let's think about the consequences of treating them equally from the beginning. How do you go about the design of an API? You have a set of requirements and you modularise it, so that you end up with a set of APIs, each of which describes the contract a class must fulfill to implement those requirements. Each API expresses one part of the original requirements. That is its purpose.

Can you say the same about the methods included in the API for testing purposes? The verify methods? No, they clearly do not express any of the "business" requirements. They have nothing to do with merging templates. What they do is express requirements that concern the technical process of producing the code that implements the business requirements. They are part of the production process not part of the product. To leave them in there and consider them to be part of the product means to mix up meta layers and to mix up the roles of producer and consumer, which is a bad idea (IMHO).

I think you would agree, if I said: All API methods have to be properly tested. If the verify methods are API methods, it follows that they too have to be properly tested. What if you need further methods to properly test the verify methods? verifyVerify()? And then verifyVerifyVerify()? Where do we stop? So it is clear that they cannot logically be part of the API they are supposed to test, otherwise you need G.W.F Hegel to bail you out there logically :-)

-Alexander

P.S: Shouldn't the merge method be public in your API?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 11:45 AM
Reply to this message Reply
> Yes, tests are a first-class client and should be allowed
> to drive the interface. HOWEVER, just as with any client,
> tests should not be allowed to make the interface bad
> (incoherent, leaky, or error prone).
>
I agree. This is why I felt better about adding the verify methods to the class instead of the get methods. The get methods could later be called by non-test clients than the verify methods. I.e., the get methods cluttered the interface and reduced the encapsulation of the class for the sake of testing. The verify methods just cluttered the interface for the sake of testing.

> If you need information your current interface isn't
> giving you in order to tell whether your object works, you
> are being informed that your current interface isn't
> adequate.
>
Actually, in this specific case I wasn't adding those verify methods to Merger to test class Merger. I added the methods to Merger to facilitate testing of all the controllers that return Mergers.

> I also believe that tests should be written in a way that
> makes them useful as "demonstrations" of typical, proper
> use.

I don't. I never look at tests to see how to use a class. I look at the JavaDoc. That's where I want the information to be. I'm curious to what extent others look at tests for figuring out how to use a class?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 11:48 AM
Reply to this message Reply
Hi Alexander,

> P.S: Shouldn't the merge method be public in your API?

Oops. You're probably right. This code is not polished for presentation as an example in an article or book. It is real code. I also noticed that we hadn't removed at least one of the get methods. This week Frank Sommers and I are trying to get the next use case out the door, and after that we'll do some post release entropy reduction.

Rick Kitts

Posts: 48
Nickname: rkitts
Registered: Jan, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 1:07 PM
Reply to this message Reply
> > I also believe that tests should be written in a way
> that
> > makes them useful as "demonstrations" of typical,
> proper
> > use.
>
> I don't. I never look at tests to see how to use a class.
> I look at the JavaDoc. That's where I want the information
> to be. I'm curious to what extent others look at tests for
> figuring out how to use a class?

Never. I do look at tests to help debug problems though. Usually I'm testing a theory by seeing if someone tested for the circumstance I think is occurring. If there's a test I probably need to reconsider. If there isn't I probably need to look closer which usually involves writing a tests.

Tim

Posts: 4
Nickname: timmorrow
Registered: Mar, 2003

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 1:09 PM
Reply to this message Reply
You're really only interested in testing that the merge() method invokes "RuntimeSingleton.getTemplate()" passing in the appropriate template name, then invokes "merge()" on the returned object passing in the appropriate context. It is the job of some other test to test the behavior Template.merge() method.

This sounds like a job for Dependency Injection + mock objects.

Here is what I usually do in this situation:

* Wrap the use of RuntimeSingleton in the merge() method with a default implementation of a new interface.

interface TemplateGetter {
Template getTemplate(String name);
}
class DefaultTemplateGetterImpl implements TemplateGetter {
Template getTemplate(String name) {
return RuntimeSingleton.getTemplate(name);
}
}

Then add a package protected setter to the class that allows you to override the DefaultTemplateGetterImpl with a different implementation.

* In your test, use jMock or similar to create a mock TemplateGetter with an expectation on getTemplate() asserting that the name matches, returning a mock Template implementation.

* Set an expectation on the mock Template that the merge() method is invoked with the appropriate Context. You'll need to define a custom jMock Constraint that essentially implements what you've already done for checking a Context.

* The jMock Constraint that you created is reusable by other unit tests.

This technique keeps all the test-oriented verification out of the actual class, without exposing the internals of the class. You do have to modify the API to add the setter for the interfaces needed for mocking.

Tim

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 3:07 PM
Reply to this message Reply
Hi Tim,

> You're really only interested in testing that the merge()
> method invokes "RuntimeSingleton.getTemplate()" passing in
> the appropriate template name, then invokes "merge()" on
> the returned object passing in the appropriate context.
> It is the job of some other test to test the behavior
> r Template.merge() method.
>
Well, I prefer to test the interface of a method when possible, not the implementation. Isn't this approach of using a mock object to check whether certain methods get invoked testing the implementation of the method?

We currently test merge by simply passing in a few simple templates and check that the templates get merged as expected. Here's one of those tests:

    public void testSimpleMerge() {
 
        Map context = new HashMap();
 
        context.put("name", "It works!");
 
        Merger merger = new Merger("helloWorld.vm", context);
 
        String s = merger.toString();
        verify(s.indexOf("It works!") != -1, s);
 
        StringWriter sw = new StringWriter();
        merger.merge(sw);
 
        verify(sw.toString().indexOf("It works!") != -1, s);
    }


And here's the template file:


<p>
Hello, $name!


As I mentioned in a previous post, I didn't add the verify methods to help me test class Merger. I added the verify methods to help me test all the controllers that generate and return trees of Mergers.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Tests First Class Clients? Posted: Jan 26, 2005 10:40 PM
Reply to this message Reply
> Ah. I've been here. I disagree more or less on a
> fundamental level. One value is in the learning. Put
> another way: If you cannot craft a test that do to input
> ranges then you probably have no contract worth speaking
> of. What I learned in writing tests is that while I might
> have thought an interface had a solid contract it didn't.
> And it didn't until I started writing tests.
>
That's interesting. In the specific case I'm talking about, I think the contracts of the controller classes I want to test are quite specific, and so is the contract of Merger. They return a tree of Mergers which should contain very well-defined values for their two constituent parts: a String template name and a context Map of name value pairs. The issue is in how I verify the constituent parts are as the contract demands.

That's where the attention shifts to class Merger itself. Except for testing, there is no need for get methods. Merger is a multi-valued return object, similar in function to the ModelAndView class of the Spring framework:

http://www.springframework.org/docs/api/org/springframework/web/servlet/ModelAndView.html

But where the spring framework's ModelAndView class is just a holder of data, Merger actually does something with the data: It performs the merge.

So to test the controllers, I either have to 1) test the output of the merge method, 2) add get methods for the template name and context map, or 3) add verify methods to class Merger.


Option 1 is possible to do. The programatic contract of merge method is simple and easy to test, just do the merge. But using the output of merge to test the controllers is a different story. That's a lot more hairy, because what comes out of that merge is really user interface. The controller output data is very buried inside of HTML and JavaScript other web stuff that changes often. It would be a lot of work to effectively screen scrape the data out of the HTML, and every time you updated the look and feel of your site, you'll risk breaking the tests of the controllers, even though it was the screen scraping that broke, not the controllers.

Option 2, add get methods, is what we did first. Verify methods in the tests classes called the get methods and did the verification.

Option 3 is what I did second, after I felt that it would be a better tradeoff to add verify methods to Merger than the get methods. But it made me feel funny, which led to me posting the issue to the weblog.

I was in China at the time, so I was upside down and that may have affected my perspective. But I got to wondering why I always think of tests as being second class citizens of my application that don't deserve any special support in the API. I want to have very good test coverage for this new architecture effort. The tests will always be there alongside the application. They are a very important part of it, so why should I feel bad about treating them like I treat other clients?

The main answer I can think of is it is good to separate these concerns, but often the concerns aren't so easy to separate. Merger's role in the world is not only to serve as a return value from controllers and to perform the actual merge. It is also there to facilitate testing of the controllers.

Joerg Gellien

Posts: 1
Nickname: jghamburg
Registered: Apr, 2003

Re: Are Tests First Class Clients? Posted: Jan 27, 2005 6:35 AM
Reply to this message Reply
> So to test the controllers, I either have to 1) test the
> output of the merge method, 2) add get methods for the
> template name and context map, or 3) add verify methods to
> class Merger.

First of all same statement as a lot of the other colleagues:
I think test classes are first class clients to a contracted api of a module/component.

Where I stumbled reading the conversation was the scope you were presenting: The Merger class.

As stated by other readers before the Merger for itself can be tested without using the public verify methods.
E.g. by using a StringWriter or using the Merger.equals() method to verify template name and context map.
The proper definition of equals() gives JUnit its strength.

Long prefix: you can test correct function of Merger by using JUnit as you displayed without "poluting" its API.

But the actual scope under test is the controller class producing a tree of Mergers as you stated.
This will provide the data required for your View. And this is the point of your actual interest.

So my structure of testing would be:
- verify that a Merger does as stated in its contract. Like you allready did.
- verify that the tree structure of Mergers is created as expected by the controler. There is no special verify neccessary on the tree elements themselves.
- verify (perhaps by using MockMergers) that the call sequence of the Mergers during traversal of the tree structure is correct.

If you are quite clear about structures / algorithms / ideas of implementation it may be enough to do a functional black box test using the contract of the controller class in use. On my behalf this does not happen too often.

Can you please provide the Controler source as well ? I am curious.

Flat View: This topic has 57 replies on 4 pages [ « | 1  2  3  4 | » ]
Topic: Why salary bonus and other incentives fail to meet their objectives Previous Topic   Next Topic Topic: Is Jikes Being Abandoned?

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use