Running a Suite

To run a suite of tests, you use the com.artima.testkit.Runner application. The application accepts command line arguments that specify an optional recipe file, runpath, zero to many Reporters, and zero to many Suite classes. Runner can be started in either of two ways. If testkit.jar is available on the classpath, then the command line takes the following form:

java [-cp <classpath>] com.artima.testkit.Runner [<recipefile>] [-p <runpath>] [reporter [...]] [-s <suiteclass> [...]]

If Runner is started using the executable jar file, the command line takes the following form:

java -jar testkit.jar [<recipefile>] [-p <runpath>] [reporter [...]] [-s <suiteclass> [...]]

A recipe file contains properties that define runpath, reporters, and suites. If a recipe file is specified, it must be the first argument. Any other arguments following the recipe file are ignored. The standard file extension for Artima Testkit recipe files is simply ".x". (You can think of them as recipes, or you can think of them as X Files.)

Specifying a Runpath

A runpath is the list of filenames, directory paths, and/or URLs that testkit uses to load classes for the running test. If runpath is specified, testkit creates a java.net.URLClassLoader to load classes available on the runpath. The graphical user interface can optionally reload the test classes for each run by creating a new URLClassLoader for each run. If the testkit JAR file is available from the Java classpath, then the classes that comprise the test may also be made available on the classpath and no runpath need be specified.

The runpath is specified with the -p option. The -p must be followed by a space, a double quote ("), a white-space-separated list of paths and URLs, and a double quote. For example:

-p "serviceuitest-1.1beta4.jar myjini http://myhost:9998/myfile.jar"

Specifying Reporters

Reporters can be specified on the command line in any of the following ways:

• -g[configs...] - causes display of a graphical user interface that allows tests to be run and results to be investigated
• -f[configs...] <filename> - causes test results to be written to the named file
• -o[configs...] - causes test results to be written to the standard output
• -e[configs...] - causes test results to be written to the standard error
• -r[configs...] <reporterclass> - causes test results to be reported to an instance of the specified fully qualified Reporter class name

The [configs...] parameter, which is used to configure reporters, is described in the next section.

The -r option causes the Reporter specified in <reporterclass> to be instantiated. Each Reporter class specified with a -r option must be public, implement com.artima.testkit.Reporter, and have a public no-arg constructor. Reporter classes must be specified with fully qualified names. If the Testkit JAR file is available on the classpath, not run directly as an executable JAR file, the specified Reporter classes can also be deployed on the classpath. If a runpath is specified with the -p option, specified Reporter classes may also be loaded from the runpath. All specified Reporter classes will be loaded and instantiated via their no-arg constructor.

For example, to run a suite using two reporters, the graphical reporter and a file reporter writing to a file named "test.out", you would type:

-g -f test.out

The -g, -o, or -e options can appear at most once each in any single command line. Multiple appearances of -f and -r result in multiple reporters unless the specified <filename> or <reporterclass> is repeated. If any of -g, -o, -e, <filename> or <reporterclass> are repeated on the command line, the Runner will print an error message and not run the tests.

Runner adds the reporters specified on the command line to a dispatch reporter, which will dispatch each method invocation on itself to each contained reporter. Runner will pass the dispatch reporter to executed suites. As a result, every specified reporter will receive every report generated by the running suite of tests. If no reporters are specified, a graphical runner will be displayed that provides a graphical report of executed suites.

Configuring Reporters

Each reporter specification on the command line can include configuration parameters. Configuration parameters are specified immediately following the -g, -o, -e, -f, or -r. Valid configuration parameters are:

• Y - report runStarting method invocations
• Z - report testStarting method invocations
• T - report testSucceeded method invocations
• F - report testFailed method invocations
• U - report suiteStarting method invocations
• P - report suiteCompleted method invocations
• B - report suiteAborted method invocations
• I - report infoProvided method invocations
• S - report runStopped method invocations
• A - report runAborted method invocations
• R - report runCompleted method invocations

Each reporter has a default configuration. If no configuration is specified on the command line for a particular reporter, that reporter uses its default configuration. Runner will configure each reporter for which configuration parameters are specified via the reporter's setConfiguration method.

For example, to run a suite using two reporters, the graphical reporter (using its default configuration) and a standard error reporter configured to print only test failures, suite aborts, and run aborts, you would type:

-g -eFBA

Note that no white space is allowed between the reporter option and the initial configuration parameters. So "-e FBA" will not work, and must be changed to "-eFBA".

Specifying Suites

Suites are specified on the command line with a -s followed by the fully qualified name of a Suite subclass, as in:

-s com.artima.serviceuitest.ServiceUITestkit

Each specified Suite class must be public, a subclass of com.artima.testkit.Suite, and contain a public no-arg constructor. Suite classes must be specified with fully qualified names. If the Testkit JAR file is available on the classpath, not run directly as an executable JAR file, the specified Suite classes can be loaded from the classpath. If a runpath is specified with the -p option, specified Suite classes may also be loaded from the runpath. All specified Suite classes will be loaded and instantiated via their no-arg constructor.

The runner will invoke execute on each instantiated com.artima.testkit.Suite, passing in the dispatch reporter to each execute method.

You can also use the -s parameter to specify JUnit test cases. To run JUnit tests from Artima Testkit, junit.jar and the test cases you wish to run must be available via the runpath. Or, if you run Artima Testkit from the class path, you may put junit.jar and your test cases on the class path. Each JUnit test case specified must be a subclass of junit.framework.TestCase, and contain a public constructor that takes a single String parameter.