Advertisement

The ScriptGenerator Application

As mentioned previously, one reason to drive tests via scripts, tables, XML documents, and so on, is that you can more easily automate the generation of script- or data-based tests than tests written by hand in Java. The ScriptGenerator application, for example, automatically generates a script that can be fed to ScriptDrivenAccountSuite.

ScriptGenerator accepts one optional command line argument, an integer number of tests commands to include in the generated script file. If no command line arguments are specified, ScriptGenerator generates 1000 test commands. ScriptGenerator writes its test script to the standard output stream.

ScriptGenerator first prints a comment indicating the script was automatically generated and specifying the number test commands it contains, and prints an initial newAccount command. Thereafter, ScriptGenerator pseudo-randomly decides whether to deposit to the Account, or withdraw from it. It pseudo-randomly selects a long amount to either deposit or withdraw, and figures out whether the action should result in an exception. It then prints an appropriate deposit or withdraw command to the standard output.

If the previously printed deposit or withdraw command is not supposed to result in an exception, ScriptGenerator prints a getBalance command to the standard output to ensure the balance is correct. If a deposit command is supposed to produce an ArithmeticException, ScriptGenerator prints a newAccount command to the standard output, because the Account contract does not promise an Account instance will still be usable after deposit throws ArithmeticException.

In this manner, ScriptGenerator generates a series of newAccount, getBalance, deposit, and withdraw commands at the standard output. As it goes, ScriptGenerator keeps track of the total number of test commands it has printed. Once it has printed the requested number of commands, the ScriptGenerator application exits.

Here is the ScriptGenerator class:

/*
 * Copyright (C) 2001-2003 Artima Software, Inc. All rights reserved.
 * Licensed under the Open Software License version 1.0.
 *
 * A copy of the Open Software License version 1.0 is available at:
 *     http://www.artima.com/suiterunner/osl10.html
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of Artima Software, Inc. For more
 * information on Artima Software, Inc., please see:
 *     http://www.artima.com/
 */
package com.artima.examples.scriptdriven.ex1;

import com.artima.examples.account.ex6.Account;
import com.artima.examples.account.ex6.InsufficientFundsException;
import org.suiterunner.Suite;
import org.suiterunner.Reporter;
import org.suiterunner.Report;

import java.io.*;
import java.util.List;
import java.util.ArrayList;
import java.util.Iterator;

/**
 * An application that generates an <code>Account</code> test script
 * containing a specified number of test commands. The number of test
 * commands is specified as the first argument to the application. If
 * no test count is specified, 1000 will be used.
 */
public class ScriptGenerator {

    private static final int DEFAULT_TEST_COUNT_MAX = 1000;

    private ScriptGenerator() {
    }

    /**
     * Application that generates an <code>Account</code> test script
     * containing the number of test commands specified as the first
     * command line argument, or 1000 test commands, if no command line
     * arguments are specified.
     */
    public static void main(String[] args) {

        int testCountMax = DEFAULT_TEST_COUNT_MAX;
        if (args.length == 1) {

            try {
                testCountMax = Integer.parseInt(args[0]);
            }
            catch (NumberFormatException e) {
                System.out.println("Must specify no args or a single integer arg "
                    + "that specifies the number of test commands to generate.");
                System.exit(1);
            }
        }
        else if (args.length != 0) {
            System.out.println("Must specify no args or a single integer arg "
                + "that specifies the number of test commands to generate.");
            System.exit(1);
        }

        // Account's contract does not promise that the Account object is still
        // valid and usable after deposit throws an ArithmeticException. This
        // app will generate a script of commands that exercises an Account by
        // making random deposits and withdrawals and verifying the balance each
        // time. If a deposit causes an ArithmeticException, the script will
        // generate a newAccount command that will cause ScriptDrivenAccountSuite
        // to create a brand new Account object for subsequent tests.

        System.out.println("# Account test script with "
            + Integer.toString(testCountMax)
            + " tests, generated by ScriptGenerator.");

        System.out.println("newAccount");

        long balance = 0;
        int testCount = 1;
        while (testCount < testCountMax) {

            // I expect randomNum to be 0 about half the
            // time and 1 the other half. Attempting to
            // set printDeposit to true half the time, which
            // will generate deposit and getBalance commands.
            // The rest of the time, printDeposit will be set to
            // false, which will result in a withdraw and
            // getBalance commands.
            int zeroOrOne = (int)(Math.random() * 2.0d);
            boolean printDeposit = false;
            if (zeroOrOne == 0) {
                printDeposit = true;
            }

            // Select a random amount to deposit or withdraw
            long amount = (long)(Math.random() * ((double)Long.MAX_VALUE));

            if (printDeposit) {

                if (balance + amount < 0L) {

                    // Overflow, so expect an ArithmethicException
                    System.out.println("deposit " + Long.toString(amount)
                        + " java.lang.ArithmeticException");

                    ++testCount;
                    if (testCount < testCountMax) {
                        System.out.println("newAccount");
                        balance = 0;
                        ++testCount;
                    }
                }
                else {

                    balance += amount;
                    System.out.println("deposit " + Long.toString(amount));
                    ++testCount;

                    if (testCount < testCountMax) {
                        System.out.println("getBalance " + Long.toString(balance));
                        ++testCount;
                    }
                }
            }
            else {

                if (balance - amount < 0L) {

                    // Overflow, so expect an InsufficientFundsException
                    System.out.println("withdraw " + Long.toString(amount)
                        + " com.artima.examples.account.ex6."
                        + "InsufficientFundsException");

                    ++testCount;

                    if (testCount < testCountMax) {
                        System.out.println("getBalance "
                            + Long.toString(balance));
                        ++testCount;
                    }
                }
                else {

                    balance -= amount;
                    System.out.println("withdraw " + Long.toString(amount));
                    ++testCount;

                    if (testCount < testCountMax) {
                        System.out.println("getBalance "
                            + Long.toString(balance));
                        ++testCount;
                    }
                }
            }
        }
    }
}

To make it easy to try ScriptDrivenAccountSuite with a script generated by ScriptGenerator, I added a sample test script file, generatedscript.txt, to the Artima SuiteRunner distribution ZIP file in version 1.0beta6. If you have a release prior to 1.0beta6, please download the latest version of Artima SuiteRunner. Once you unzip the distribution ZIP file, you'll find generatedscript.txt in the suiterunner-[release] directory.

generatedscript.txt, which was generated by ScriptGenerator, contains 1000 test commands. Here are the first 25 lines of generatedscript.txt:

# Account test script with 1000 tests, generated by ScriptGenerator.
newAccount
deposit 2204897606719931392
getBalance 2204897606719931392
withdraw 6699598046204537856 com.artima.examples.account.ex6.InsufficientFundsException
getBalance 2204897606719931392
deposit 4194476087876486144
getBalance 6399373694596417536
withdraw 4864310293971913728
getBalance 1535063400624503808
deposit 1584119828864405504
getBalance 3119183229488909312
deposit 5104144951917947904
getBalance 8223328181406857216
deposit 6709782093051939840 java.lang.ArithmeticException
newAccount
withdraw 6341517708414168064 com.artima.examples.account.ex6.InsufficientFundsException
getBalance 0
withdraw 7335749506867528704 com.artima.examples.account.ex6.InsufficientFundsException
getBalance 0
withdraw 5993124522139121664 com.artima.examples.account.ex6.InsufficientFundsException
getBalance 0
deposit 2149650782832521216
getBalance 2149650782832521216
deposit 1414786779105710080

To execute ScriptDrivenAccountSuite using generatedscript.txt, run the following command from the suiterunner-[release] subdirectory of the directory in which you unzipped the Artima SuiteRunner distribution ZIP file:

java -DScriptFile=generatedscript.txt -jar suiterunner-1.0beta6.jar accountscript.srj

When you execute the previous command, you should see results similar to those shown in Figure 3. Because the previous command sets ScriptFile system property, ScriptDrivenAccountSuite will use the specified script file name, generatedscript.txt. This script file, which sits alongside scriptdriven.srj, contains 1000 tests, 331 of which fail. generatescript.txt's 331 failures, like the six failures produced by script.txt are caused by a bug introduced intentionally into class Account to demonstrate how Artima SuiteRunner reports failed tests. If you fix this bug and rerun the previous command, you should get 1000 successful tests and a green bar.

Figure 3. ScriptDrivenAccountSuite results for script generatedscript.txt

Get Help in the SuiteRunner Forum

For help with Artima SuiteRunner, please post to the SuiteRunner Forum.

About the Author

Bill Venners is President of Artima Software, Inc. and Editor-In-Chief of Artima.com. He is the author of Inside the Java Virtual Machine (Computing McGraw-Hill), a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard for associating user interfaces to Jini services. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community.

Resources

For more information about defining test methods, see the Artima SuiteRunner Tutorial:
http://www.artima.com/suiterunner/tutorial.html

Why We Refactored JUnit
http://www.artima.com/suiterunner/why.html

Artima SuiteRunner Tutorial, Building Conformance and Unit Tests with Artima SuiteRunner:
http://www.artima.com/suiterunner/tutorial.html

Getting Started with Artima SuiteRunner, How to Run the Simple Example Included in the Distribution:
http://www.artima.com/suiterunner/start.html

Runnning JUnit Tests with Artima SuiteRunner, how to use Artima SuiteRunner as a JUnit runner to run your existing JUnit test suites:
http://www.artima.com/suiterunner/junit.html

Artima SuiteRunner home page:
http://www.artima.com/suiterunner/index.html

Artima SuiteRunner download page (You must log onto Artima.com to download the release):
http://www.artima.com/suiterunner/download.jsp

The SuiteRunner Forum:
http://www.artima.com/forums/forum.jsp?forum=61