|
|
|
|
Artima SuiteRunner |
Why |
Getting Started |
Tutorial |
Get Help |
Discuss |
Print |
Email |
First Page |
Previous |
Next
|
|
Advertisement
|
main method
of class org.suiterunner.Runner. Runner.main parses
command line arguments and runs a suite of tests, optionally firing up the GUI.
The method I extracted, parseArgsIntoLists, does part of the
work of parsing of the command line arguments to the SuiteRunner application. Now, to test the public
method that calls this private method, I would need to test
main. Main, of course, is the entire application, which makes the
method rather difficult to test. In fact, I have no existing test for
main.
At this point, you may be wondering, if I were writing tests first in the style of test-driven development [10], how did I ever end up writing parsing code that had no unit tests? The main reason is that my test infection has come in stages. I did actually catch a unit test flu long before I had heard of JUnit or read Test Infected [11]. Back when I was building Windows applications in C++, for example, I would write a bit of code to test a newly implemented method, then execute that code and watch it execute by stepping through the method under test with the debugger. This kind of unit testing did help me achieve robustness, but the tests themselves didn't check for the proper behavior. I checked for the proper behavior myself by observing via the debugger. The tests weren't automated, and therefore I didn't save them so they could be run again later. When I read Test Infected, I immediately saw the value of automating the tests and keeping them around as a kind of regression test after refactoring, but for a long time it didn't make sense to me to write the tests first. I wanted to write the tests after I implemented the functionality, because that was when I had run the tests with the debugger. A secondary reason I didn't write tests first while developing much of SuiteRunner is that I wanted to write SuiteRunner's tests with SuiteRunner itself, in an effort to eat my own dog food. Until SuiteRunner's basic API settled, I didn't have the testing toolkit I wanted to use to write the tests.
Since that time, however, the testing virus has taken a stronger hold on me, and I do now
prefer to write unit tests first most of the time. I prefer to write tests first not so much
because I find I end up with cleaner designs, which is usually promoted as the
main benefit of test-driven development. Rather, I prefer to write tests first because
I find that often if I dive into the code under pressure, with the intent that
I'll write the test later, the test doesn't actually ever get written.
SuiteRunner itself has very few unit tests at this point for that very reason.
Here's the parseArgsIntoLists method:
private static void parseArgsIntoLists(String[] args, List runpathList,
List reportersList, List suitesList) {
if (args == null || runpathList == null
|| reportersList == null || suitesList == null) {
throw new NullPointerException();
}
for (int i = 0; i < args.length; i++) {
if (args[i].startsWith("-p")) {
runpathList.add(args[i]);
runpathList.add(args[i + 1]);
++i;
}
else if (args[i].startsWith("-g")) {
reportersList.add(args[i]);
}
else if (args[i].startsWith("-o")) {
reportersList.add(args[i]);
}
else if (args[i].startsWith("-e")) {
reportersList.add(args[i]);
}
else if (args[i].startsWith("-f")) {
reportersList.add(args[i]);
reportersList.add(args[i + 1]);
++i;
}
else if (args[i].startsWith("-r")) {
reportersList.add(args[i]);
reportersList.add(args[i + 1]);
++i;
}
else if (args[i].startsWith("-s")) {
suitesList.add(args[i]);
do {
++i;
suitesList.add(args[i]);
} while (i + 1 < args.length);
}
else {
throw new IllegalArgumentException("Unrecognized argument: "
+ args[i]);
}
}
}
The command line for SuiteRunner contains three types of information used by
SuiteRunner to run tests: runpath, reporters, and suites. The parseArgsIntoLists
method merely goes through the arguments passed as an array of Strings,
and places each argument into one of the lists, runpathList,
reportersList, and suitesList.
Before I write a test for this private method, I would ask whether my urge
to write this unit test represents a code smell, as Charles Miller put it in his weblog? Does it
indicate that parseArgsIntoLists should be moved into another
class to promote reusability, as the JUnit FAQ suggests? Would Dave and Andy
say its a warning sign that there's another class in there struggling to get
out? Well, maybe. I could concievably create an ArgumentsParser class that only holds a
few static methods that perform the parsing work. Both the
ArgumentsParser class and the methods it contains could be
package access, which would make them easy to test. But that just doesn't
feel right to me. These methods are only called by Runner.main.
They clearly feel like private methods to me. The only reason I would be
moving them to an ArgumentsParser class is to be able to test
them. I would in fact be using approach number 2: make the private methods
package access.
Instead, for this example I decided to take approach number 4, and use reflection. I looked at
both Vladimir Bossicard's junitx.utils.PrivateAccessor and
Charlie Hubbard and Prashant Dhotke's PrivilegedAccessor, but
decided that neither of them helped me quite the way I wanted.
For one thing, these classes both have the ability to test fields to make sure
they are set correctly. As yet I have never felt any urge to directly access
private fields from unit tests. I just want to be able to test private utility
methods. The main problem I had with these two classes, however, is how they
dealt with the exceptions that may be thrown when trying to invoke the private
method via reflection. Each class has one or more methods whose job it is
invoke a method with reflection. PrivilegedAccessor's two invokeMethod
methods passes any exception back to its caller, including three checked exceptions declared in
the throws clause: NoSuchMethodException, IllegalAccessException,
and InvocationTargetException. By contrast, PrivateAccessor's two
invoke methods catch InvocationTargetException, and extract and throw the
target exception, the actual exception thrown by the invoked method. It then catches any other exception, and
throws NoSuchMethodException. I didn't like that the caller of PrivilegedAccessor.invokeMethod
would always need to handle the three checked exceptions, because I figured the general way to handle any
exception would be to let the test fail. I was also concerned that the PrivateAccessor.invoke
was throwing away potentially useful stack trace information in its exception handling policy. What I really wanted was a method that attempted
to invoke a private method with reflection, which wrapped any thrown exception besides InvocationTargetException in
an unchecked TestFailedException. Most of the time this exception would cause the test to fail. In tests that
were expecting an exception to be thrown, the exception contained in InvocationTargetException could be extracted and tested for correctness.
Therefore, I wrote invokeStaticMethod. The setAccessible(true) call is what enables the private method to be invoked from outside the
class. An corresponding invokeStaticMethod implementation for use with JUnit would throw AssertionFailedError rather than
TestFailedException. Here's the code:
private static void invokeStaticMethod(Class targetClass,
String methodName, Class[] argClasses, Object[] argObjects)
throws InvocationTargetException {
try {
Method method = targetClass.getDeclaredMethod(methodName,
argClasses);
method.setAccessible(true);
method.invoke(null, argObjects);
}
catch (NoSuchMethodException e) {
// Should happen only rarely, because most times the
// specified method should exist. If it does happen, just let
// the test fail so the programmer can fix the problem.
throw new TestFailedException(e);
}
catch (SecurityException e) {
// Should happen only rarely, because the setAccessible(true)
// should be allowed in when running unit tests. If it does
// happen, just let the test fail so the programmer can fix
// the problem.
throw new TestFailedException(e);
}
catch (IllegalAccessException e) {
// Should never happen, because setting accessible flag to
// true. If setting accessible fails, should throw a security
// exception at that point and never get to the invoke. But
// just in case, wrap it in a TestFailedException and let a
// human figure it out.
throw new TestFailedException(e);
}
catch (IllegalArgumentException e) {
// Should happen only rarely, because usually the right
// number and types of arguments will be passed. If it does
// happen, just let the test fail so the programmer can fix
// the problem.
throw new TestFailedException(e);
}
}
Next, I created a convenience method that invokes the particular private method I wanted to test:
private static void invokeParseArgsIntoLists(String[] args,
List runpathList, List reportersList, List suitesList)
throws InvocationTargetException {
// Purposely pass null values to the method, to make sure it throws
// NullPointerException
Class[] argClasses = {String[].class, List.class, List.class, List.class };
Object[] argObjects = {args, runpathList, reportersList, suitesList };
invokeStaticMethod(Runner.class, "parseArgsIntoLists", argClasses, argObjects);
}
At last, I could write tests against the private method without too much excess clutter, like this:
public void testParseArgsIntoLists() throws InvocationTargetException {
String[] args = new String[0];
List runpathList = new ArrayList();
List reportersList = new ArrayList();
List suitesList = new ArrayList();
try {
invokeParseArgsIntoLists(null, runpathList, reportersList, suitesList);
fail();
}
catch (InvocationTargetException e) {
// throw the InvocationTargetException unless the target
// exception is NullPointerException, which is expected
Throwable targetException = e.getTargetException();
if (!(targetException instanceof NullPointerException)) {
throw e;
}
}
try {
invokeParseArgsIntoLists(args, null, reportersList, suitesList);
fail();
}
catch (InvocationTargetException e) {
// throw the InvocationTargetException unless the target
// exception is NullPointerException, which is expected
Throwable targetException = e.getTargetException();
if (!(targetException instanceof NullPointerException)) {
throw e;
}
}
try {
invokeParseArgsIntoLists(args, runpathList, null, suitesList);
fail();
}
catch (InvocationTargetException e) {
// throw the InvocationTargetException unless the target
// exception is NullPointerException, which is expected
Throwable targetException = e.getTargetException();
if (!(targetException instanceof NullPointerException)) {
throw e;
}
}
try {
invokeParseArgsIntoLists(args, runpathList, reportersList, null);
fail();
}
catch (InvocationTargetException e) {
// throw the InvocationTargetException unless the target
// exception is NullPointerException, which is expected
Throwable targetException = e.getTargetException();
if (!(targetException instanceof NullPointerException)) {
throw e;
}
}
args = new String[7];
args[0] = "-p";
args[1] = "\"mydir\"";
args[2] = "-g";
args[3] = "-f";
args[4] = "test.out";
args[5] = "-s";
args[6] = "MySuite";
runpathList.clear();
reportersList.clear();
suitesList.clear();
invokeParseArgsIntoLists(args, runpathList, reportersList,
suitesList);
verify(runpathList.size() == 2);
verify(runpathList.get(0).equals(args[0]));
verify(runpathList.get(1).equals(args[1]));
verify(reportersList.size() == 3);
verify(reportersList.get(0).equals(args[2]));
verify(reportersList.get(1).equals(args[3]));
verify(reportersList.get(2).equals(args[4]));
verify(suitesList.size() == 2);
verify(suitesList.get(0).equals(args[5]));
verify(suitesList.get(1).equals(args[6]));
args = new String[9];
args[0] = "-p";
args[1] = "\"mydir\"";
args[2] = "-e";
args[3] = "-o";
args[4] = "-r";
args[5] = "MyCustomReporter";
args[6] = "-s";
args[7] = "MySuite";
args[8] = "MyOtherSuite";
runpathList.clear();
reportersList.clear();
suitesList.clear();
invokeParseArgsIntoLists(args, runpathList, reportersList,
suitesList);
verify(runpathList.size() == 2);
verify(runpathList.get(0).equals(args[0]));
verify(runpathList.get(1).equals(args[1]));
verify(reportersList.size() == 4);
verify(reportersList.get(0).equals(args[2]));
verify(reportersList.get(1).equals(args[3]));
verify(reportersList.get(2).equals(args[4]));
verify(reportersList.get(3).equals(args[5]));
verify(suitesList.size() == 3);
verify(suitesList.get(0).equals(args[6]));
verify(suitesList.get(1).equals(args[7]));
verify(suitesList.get(2).equals(args[8]));
args = new String[10];
args[0] = "-p";
args[1] = "\"serviceuitest-1.1beta4.jar myjini http://myhost:9998/myfile.jar\"";
args[2] = "-g";
args[3] = "-s";
args[4] = "MySuite";
args[5] = "MySecondSuite";
args[6] = "MyThirdSuite";
args[7] = "MyFourthSuite";
args[8] = "MyFifthSuite";
args[9] = "MySixthSuite";
runpathList.clear();
reportersList.clear();
suitesList.clear();
invokeParseArgsIntoLists(args, runpathList, reportersList,
suitesList);
verify(runpathList.size() == 2);
verify(runpathList.get(0).equals(args[0]));
verify(runpathList.get(1).equals(args[1]));
verify(reportersList.size() == 1);
verify(reportersList.get(0).equals(args[2]));
verify(suitesList.size() == 7);
verify(suitesList.get(0).equals(args[3]));
verify(suitesList.get(1).equals(args[4]));
verify(suitesList.get(2).equals(args[5]));
verify(suitesList.get(3).equals(args[6]));
verify(suitesList.get(4).equals(args[7]));
verify(suitesList.get(5).equals(args[8]));
verify(suitesList.get(6).equals(args[9]));
}
Approach 1, testing private methods indirectly by testing the package-level, protected, and public methods that call them, will often be the best approach. In cases where you really do want to test private methods directly, using reflection to test private methods, although rather cumbersome, does provide the cleanest separation of test code from production code, and the least impact on production code. However, if you don't mind making those particular private methods you want to test package access, you could use approach 2. Or if you don't mind placing a nested test class inside your production class under test, approach 3 would at least let you keep the private methods private.
There isn't a perfect answer. But if you adopt approach 4, you will ultimately end up with a handful of methods
like invokeStaticMethod that you can reuse. Once you write a
convenience method, like invokeParseArgsIntoLists, for a private method, you can write tests
against the private method without much difficulty.
Do you ever test private methods? If so, what is your preferred apprach? Discuss this article in the Articles Forum topic, Testing Private Methods with JUnit and SuiteRunner.
Bill Venners is President of
Artima Software, Inc. and Editor-In-Chief of Artima.com. He is the author of Inside the Java Virtual Machine
(Computing McGraw-Hill), a programmer-oriented survey of the Java platform's
architecture and internals. His popular columns in JavaWorld magazine
covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini
Community's ServiceUI project, whose
ServiceUI API became the de facto standard for associating user interfaces to Jini
services. Bill also serves as an elected member of the Jini Community's initial
Technical Oversight Committee (TOC), and in this role helped to define the
governance process for the community.
Resources
1. The ServiceUI API defines a standard way to attach user interfaces to Jini services:
http://www.artima.com/jini/serviceui/index.html
2. Daniel Steinberg is currently the Editor-In-Chief of Java.NET:
http://www.java.net/
3. Artima SuiteRunner is a free open source testing toolkit and JUnit runner:
http://www.artima.com/suiterunner/index.html
4.JUnit FAQ question about testing private methods:
http://junit.sourceforge.net/doc/faq/faq.htm#tests_10
5. Testing Private Methods (Don't Do It), a weblog post by Charles Miller:
http://fishbowl.pastiche.org/2003/03/28/testing_private_methods_dont_do_it
6. Andy Hunt and Dave Thomas are the authors of Pragmatic Unit Testing, which is available at The Pragmatic Store.
7. JUnit Addons is a collection of helper classes for JUnit created by Vladimar R. Bossicard:
http://sourceforge.net/projects/junit-addons
8. PrivateAccessor is the class from JUnit Addons that facilates testing private members:
http://junit-addons.sourceforge.net/junitx/util/PrivateAccessor.html
9.PrivilegedAccessor class, which you can use to access private members:
http://groups.yahoo.com/group/junit/files/src/PrivilegedAccessor.java
10. Test Driven Development by Example, by Kent Beck, describes the test-first technique:
http://www.amazon.com/exec/obidos/ASIN/0321146530/
11. Test Infected, by Kent Beck and Erich Gamma, introduced JUnit to the world:
http://junit.sourceforge.net/doc/testinfected/testing.htm
Unit Testing Private Methods, a weblog post about nUnit by Ted Graham:
http://weblogs.asp.net/tgraham/archive/2003/12/31/46984.aspx
Subverting Java's Access Protection for Unit Testing, an O'Reilly OnJava.com article by Ross Burton:
http://www.onjava.com/pub/a/onjava/2003/11/12/reflection.html
Class RunnerSuite, from which the code snippets in this article were taken, appears in its entirety here:
http://www.artima.com/suiterunner/privateExample.html
Why We Refactored JUnit
http://www.artima.com/suiterunner/why.html
Artima SuiteRunner Tutorial,
Building Conformance and Unit Tests with Artima SuiteRunner:
http://www.artima.com/suiterunner/tutorial.html
Getting Started with Artima SuiteRunner,
How to Run the Simple Example Included in the Distribution:
http://www.artima.com/suiterunner/start.html
Runnning JUnit Tests with Artima SuiteRunner,
how to use Artima SuiteRunner as a JUnit runner to run your existing JUnit test suites:
http://www.artima.com/suiterunner/junit.html
Artima SuiteRunner home page:
http://www.artima.com/suiterunner/index.html
Artima SuiteRunner download page (You must log onto Artima.com to download the release):
http://www.artima.com/suiterunner/download.jsp
The SuiteRunner Forum:
http://www.artima.com/forums/forum.jsp?forum=61
|
Sponsored Links
|