The Artima Developer Community
Sponsored Link

Testing Private Methods with JUnit and SuiteRunner
by Bill Venners
May 24, 2004

<<  Page 3 of 3

Advertisement

An Example

To give one example of a private method that, in my opinion, merits direct unit testing, I extracted some functionality out of the main method of class org.suiterunner.Runner. Runner.main parses command line arguments and runs a suite of tests, optionally firing up the GUI. The method I extracted, parseArgsIntoLists, does part of the work of parsing of the command line arguments to the SuiteRunner application. Now, to test the public method that calls this private method, I would need to test main. Main, of course, is the entire application, which makes the method rather difficult to test. In fact, I have no existing test for main.

At this point, you may be wondering, if I were writing tests first in the style of test-driven development [10], how did I ever end up writing parsing code that had no unit tests? The main reason is that my test infection has come in stages. I did actually catch a unit test flu long before I had heard of JUnit or read Test Infected [11]. Back when I was building Windows applications in C++, for example, I would write a bit of code to test a newly implemented method, then execute that code and watch it execute by stepping through the method under test with the debugger. This kind of unit testing did help me achieve robustness, but the tests themselves didn't check for the proper behavior. I checked for the proper behavior myself by observing via the debugger. The tests weren't automated, and therefore I didn't save them so they could be run again later. When I read Test Infected, I immediately saw the value of automating the tests and keeping them around as a kind of regression test after refactoring, but for a long time it didn't make sense to me to write the tests first. I wanted to write the tests after I implemented the functionality, because that was when I had run the tests with the debugger. A secondary reason I didn't write tests first while developing much of SuiteRunner is that I wanted to write SuiteRunner's tests with SuiteRunner itself, in an effort to eat my own dog food. Until SuiteRunner's basic API settled, I didn't have the testing toolkit I wanted to use to write the tests.

Since that time, however, the testing virus has taken a stronger hold on me, and I do now prefer to write unit tests first most of the time. I prefer to write tests first not so much because I find I end up with cleaner designs, which is usually promoted as the main benefit of test-driven development. Rather, I prefer to write tests first because I find that often if I dive into the code under pressure, with the intent that I'll write the test later, the test doesn't actually ever get written. SuiteRunner itself has very few unit tests at this point for that very reason. Here's the parseArgsIntoLists method:

    private static void parseArgsIntoLists(String[] args, List runpathList,
            List reportersList, List suitesList) {

        if (args == null || runpathList == null
                || reportersList == null || suitesList == null) {

            throw new NullPointerException();
        }

        for (int i = 0; i < args.length; i++) {

            if (args[i].startsWith("-p")) {

                runpathList.add(args[i]);
                runpathList.add(args[i + 1]);
                ++i;
            }
            else if (args[i].startsWith("-g")) {

                reportersList.add(args[i]);
            }
            else if (args[i].startsWith("-o")) {

                reportersList.add(args[i]);
            }
            else if (args[i].startsWith("-e")) {

                reportersList.add(args[i]);
            }
            else if (args[i].startsWith("-f")) {

                reportersList.add(args[i]);
                reportersList.add(args[i + 1]);
                ++i;
            }
            else if (args[i].startsWith("-r")) {

                reportersList.add(args[i]);
                reportersList.add(args[i + 1]);
                ++i;
            }
            else if (args[i].startsWith("-s")) {

                suitesList.add(args[i]);
                do  {
                    ++i;
                    suitesList.add(args[i]);
                } while (i + 1 < args.length);
            }
            else {
                throw new IllegalArgumentException("Unrecognized argument: "
                        + args[i]);
            }
        }
    }

The command line for SuiteRunner contains three types of information used by SuiteRunner to run tests: runpath, reporters, and suites. The parseArgsIntoLists method merely goes through the arguments passed as an array of Strings, and places each argument into one of the lists, runpathList, reportersList, and suitesList.

Before I write a test for this private method, I would ask whether my urge to write this unit test represents a code smell, as Charles Miller put it in his weblog? Does it indicate that parseArgsIntoLists should be moved into another class to promote reusability, as the JUnit FAQ suggests? Would Dave and Andy say its a warning sign that there's another class in there struggling to get out? Well, maybe. I could concievably create an ArgumentsParser class that only holds a few static methods that perform the parsing work. Both the ArgumentsParser class and the methods it contains could be package access, which would make them easy to test. But that just doesn't feel right to me. These methods are only called by Runner.main. They clearly feel like private methods to me. The only reason I would be moving them to an ArgumentsParser class is to be able to test them. I would in fact be using approach number 2: make the private methods package access.

Instead, for this example I decided to take approach number 4, and use reflection. I looked at both Vladimir Bossicard's junitx.utils.PrivateAccessor and Charlie Hubbard and Prashant Dhotke's PrivilegedAccessor, but decided that neither of them helped me quite the way I wanted. For one thing, these classes both have the ability to test fields to make sure they are set correctly. As yet I have never felt any urge to directly access private fields from unit tests. I just want to be able to test private utility methods. The main problem I had with these two classes, however, is how they dealt with the exceptions that may be thrown when trying to invoke the private method via reflection. Each class has one or more methods whose job it is invoke a method with reflection. PrivilegedAccessor's two invokeMethod methods passes any exception back to its caller, including three checked exceptions declared in the throws clause: NoSuchMethodException, IllegalAccessException, and InvocationTargetException. By contrast, PrivateAccessor's two invoke methods catch InvocationTargetException, and extract and throw the target exception, the actual exception thrown by the invoked method. It then catches any other exception, and throws NoSuchMethodException. I didn't like that the caller of PrivilegedAccessor.invokeMethod would always need to handle the three checked exceptions, because I figured the general way to handle any exception would be to let the test fail. I was also concerned that the PrivateAccessor.invoke was throwing away potentially useful stack trace information in its exception handling policy. What I really wanted was a method that attempted to invoke a private method with reflection, which wrapped any thrown exception besides InvocationTargetException in an unchecked TestFailedException. Most of the time this exception would cause the test to fail. In tests that were expecting an exception to be thrown, the exception contained in InvocationTargetException could be extracted and tested for correctness.

Therefore, I wrote invokeStaticMethod. The setAccessible(true) call is what enables the private method to be invoked from outside the class. An corresponding invokeStaticMethod implementation for use with JUnit would throw AssertionFailedError rather than TestFailedException. Here's the code:

    private static void invokeStaticMethod(Class targetClass,
            String methodName, Class[] argClasses, Object[] argObjects)
            throws InvocationTargetException {

        try {
            Method method = targetClass.getDeclaredMethod(methodName,
                    argClasses);
            method.setAccessible(true);
            method.invoke(null, argObjects);
        }
        catch (NoSuchMethodException e) {
            // Should happen only rarely, because most times the
            // specified method should exist. If it does happen, just let
            // the test fail so the programmer can fix the problem.
            throw new TestFailedException(e);
        }
        catch (SecurityException e) {
            // Should happen only rarely, because the setAccessible(true)
            // should be allowed in when running unit tests. If it does
            // happen, just let the test fail so the programmer can fix
            // the problem.
            throw new TestFailedException(e);
        }
        catch (IllegalAccessException e) {
            // Should never happen, because setting accessible flag to
            // true. If setting accessible fails, should throw a security
            // exception at that point and never get to the invoke. But
            // just in case, wrap it in a TestFailedException and let a
            // human figure it out.
            throw new TestFailedException(e);
        }
        catch (IllegalArgumentException e) {
            // Should happen only rarely, because usually the right
            // number and types of arguments will be passed. If it does
            // happen, just let the test fail so the programmer can fix
            // the problem.
            throw new TestFailedException(e);
        }
    }

Next, I created a convenience method that invokes the particular private method I wanted to test:

    private static void invokeParseArgsIntoLists(String[] args,
            List runpathList, List reportersList, List suitesList)
            throws InvocationTargetException {

        // Purposely pass null values to the method, to make sure it throws
        // NullPointerException

        Class[] argClasses = {String[].class, List.class, List.class, List.class };
        Object[] argObjects = {args, runpathList, reportersList, suitesList };

        invokeStaticMethod(Runner.class, "parseArgsIntoLists", argClasses, argObjects);
    }

At last, I could write tests against the private method without too much excess clutter, like this:

    public void testParseArgsIntoLists() throws InvocationTargetException {

        String[] args = new String[0];
        List runpathList = new ArrayList();
        List reportersList = new ArrayList();
        List suitesList = new ArrayList();

        try {
            invokeParseArgsIntoLists(null, runpathList, reportersList, suitesList);
            fail();
        }
        catch (InvocationTargetException e) {

            // throw the InvocationTargetException unless the target
            // exception is NullPointerException, which is expected
            Throwable targetException = e.getTargetException();
            if (!(targetException instanceof NullPointerException)) {
                throw e;
            }
        }

        try {
            invokeParseArgsIntoLists(args, null, reportersList, suitesList);
            fail();
        }
        catch (InvocationTargetException e) {

            // throw the InvocationTargetException unless the target
            // exception is NullPointerException, which is expected
            Throwable targetException = e.getTargetException();
            if (!(targetException instanceof NullPointerException)) {
                throw e;
            }
        }

        try {
            invokeParseArgsIntoLists(args, runpathList, null, suitesList);
            fail();
        }
        catch (InvocationTargetException e) {

            // throw the InvocationTargetException unless the target
            // exception is NullPointerException, which is expected
            Throwable targetException = e.getTargetException();
            if (!(targetException instanceof NullPointerException)) {
                throw e;
            }
        }

        try {
            invokeParseArgsIntoLists(args, runpathList, reportersList, null);
            fail();
        }
        catch (InvocationTargetException e) {

            // throw the InvocationTargetException unless the target
            // exception is NullPointerException, which is expected
            Throwable targetException = e.getTargetException();
            if (!(targetException instanceof NullPointerException)) {
                throw e;
            }
        }

        args = new String[7];
        args[0] = "-p";
        args[1] = "\"mydir\"";
        args[2] = "-g";
        args[3] = "-f";
        args[4] = "test.out";
        args[5] = "-s";
        args[6] = "MySuite";
        runpathList.clear();
        reportersList.clear();
        suitesList.clear();

        invokeParseArgsIntoLists(args, runpathList, reportersList,
                suitesList);

        verify(runpathList.size() == 2);
        verify(runpathList.get(0).equals(args[0]));
        verify(runpathList.get(1).equals(args[1]));

        verify(reportersList.size() == 3);
        verify(reportersList.get(0).equals(args[2]));
        verify(reportersList.get(1).equals(args[3]));
        verify(reportersList.get(2).equals(args[4]));

        verify(suitesList.size() == 2);
        verify(suitesList.get(0).equals(args[5]));
        verify(suitesList.get(1).equals(args[6]));

        args = new String[9];
        args[0] = "-p";
        args[1] = "\"mydir\"";
        args[2] = "-e";
        args[3] = "-o";
        args[4] = "-r";
        args[5] = "MyCustomReporter";
        args[6] = "-s";
        args[7] = "MySuite";
        args[8] = "MyOtherSuite";
        runpathList.clear();
        reportersList.clear();
        suitesList.clear();

        invokeParseArgsIntoLists(args, runpathList, reportersList,
                suitesList);

        verify(runpathList.size() == 2);
        verify(runpathList.get(0).equals(args[0]));
        verify(runpathList.get(1).equals(args[1]));

        verify(reportersList.size() == 4);
        verify(reportersList.get(0).equals(args[2]));
        verify(reportersList.get(1).equals(args[3]));
        verify(reportersList.get(2).equals(args[4]));
        verify(reportersList.get(3).equals(args[5]));

        verify(suitesList.size() == 3);
        verify(suitesList.get(0).equals(args[6]));
        verify(suitesList.get(1).equals(args[7]));
        verify(suitesList.get(2).equals(args[8]));

        args = new String[10];
        args[0] = "-p";
        args[1] = "\"serviceuitest-1.1beta4.jar myjini http://myhost:9998/myfile.jar\"";
        args[2] = "-g";
        args[3] = "-s";
        args[4] = "MySuite";
        args[5] = "MySecondSuite";
        args[6] = "MyThirdSuite";
        args[7] = "MyFourthSuite";
        args[8] = "MyFifthSuite";
        args[9] = "MySixthSuite";
        runpathList.clear();
        reportersList.clear();
        suitesList.clear();

        invokeParseArgsIntoLists(args, runpathList, reportersList,
                suitesList);

        verify(runpathList.size() == 2);
        verify(runpathList.get(0).equals(args[0]));
        verify(runpathList.get(1).equals(args[1]));

        verify(reportersList.size() == 1);
        verify(reportersList.get(0).equals(args[2]));

        verify(suitesList.size() == 7);
        verify(suitesList.get(0).equals(args[3]));
        verify(suitesList.get(1).equals(args[4]));
        verify(suitesList.get(2).equals(args[5]));
        verify(suitesList.get(3).equals(args[6]));
        verify(suitesList.get(4).equals(args[7]));
        verify(suitesList.get(5).equals(args[8]));
        verify(suitesList.get(6).equals(args[9]));
    }

Conclusion

Approach 1, testing private methods indirectly by testing the package-level, protected, and public methods that call them, will often be the best approach. In cases where you really do want to test private methods directly, using reflection to test private methods, although rather cumbersome, does provide the cleanest separation of test code from production code, and the least impact on production code. However, if you don't mind making those particular private methods you want to test package access, you could use approach 2. Or if you don't mind placing a nested test class inside your production class under test, approach 3 would at least let you keep the private methods private.

There isn't a perfect answer. But if you adopt approach 4, you will ultimately end up with a handful of methods like invokeStaticMethod that you can reuse. Once you write a convenience method, like invokeParseArgsIntoLists, for a private method, you can write tests against the private method without much difficulty.

Talk Back!

Do you ever test private methods? If so, what is your preferred apprach? Discuss this article in the Articles Forum topic, Testing Private Methods with JUnit and SuiteRunner.

About the Author

Bill Venners is President of Artima Software, Inc. and Editor-In-Chief of Artima.com. He is the author of Inside the Java Virtual Machine (Computing McGraw-Hill), a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard for associating user interfaces to Jini services. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community.

Resources

1. The ServiceUI API defines a standard way to attach user interfaces to Jini services:
http://www.artima.com/jini/serviceui/index.html

2. Daniel Steinberg is currently the Editor-In-Chief of Java.NET:
http://www.java.net/

3. Artima SuiteRunner is a free open source testing toolkit and JUnit runner:
http://www.artima.com/suiterunner/index.html

4.JUnit FAQ question about testing private methods:
http://junit.sourceforge.net/doc/faq/faq.htm#tests_10

5. Testing Private Methods (Don't Do It), a weblog post by Charles Miller:
http://fishbowl.pastiche.org/2003/03/28/testing_private_methods_dont_do_it

6. Andy Hunt and Dave Thomas are the authors of Pragmatic Unit Testing, which is available at The Pragmatic Store.

7. JUnit Addons is a collection of helper classes for JUnit created by Vladimar R. Bossicard:
http://sourceforge.net/projects/junit-addons

8. PrivateAccessor is the class from JUnit Addons that facilates testing private members:
http://junit-addons.sourceforge.net/junitx/util/PrivateAccessor.html

9.PrivilegedAccessor class, which you can use to access private members:
http://groups.yahoo.com/group/junit/files/src/PrivilegedAccessor.java

10. Test Driven Development by Example, by Kent Beck, describes the test-first technique:
http://www.amazon.com/exec/obidos/ASIN/0321146530/

11. Test Infected, by Kent Beck and Erich Gamma, introduced JUnit to the world:
http://junit.sourceforge.net/doc/testinfected/testing.htm

Unit Testing Private Methods, a weblog post about nUnit by Ted Graham:
http://weblogs.asp.net/tgraham/archive/2003/12/31/46984.aspx

Subverting Java's Access Protection for Unit Testing, an O'Reilly OnJava.com article by Ross Burton:
http://www.onjava.com/pub/a/onjava/2003/11/12/reflection.html

Class RunnerSuite, from which the code snippets in this article were taken, appears in its entirety here:
http://www.artima.com/suiterunner/privateExample.html

Why We Refactored JUnit
http://www.artima.com/suiterunner/why.html

Artima SuiteRunner Tutorial, Building Conformance and Unit Tests with Artima SuiteRunner:
http://www.artima.com/suiterunner/tutorial.html

Getting Started with Artima SuiteRunner, How to Run the Simple Example Included in the Distribution:
http://www.artima.com/suiterunner/start.html

Runnning JUnit Tests with Artima SuiteRunner, how to use Artima SuiteRunner as a JUnit runner to run your existing JUnit test suites:
http://www.artima.com/suiterunner/junit.html

Artima SuiteRunner home page:
http://www.artima.com/suiterunner/index.html

Artima SuiteRunner download page (You must log onto Artima.com to download the release):
http://www.artima.com/suiterunner/download.jsp

The SuiteRunner Forum:
http://www.artima.com/forums/forum.jsp?forum=61

<<  Page 3 of 3


Sponsored Links



Google
  Web Artima.com   
Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us