|
|
|
ScalaTest is a free, open-source testing tool for Scala and Java programmers. It is written in Scala, and enables you to write tests in Scala to test either Scala or Java code. It is released under the Apache 2.0 open source license.
The current release of ScalaTest is version 0.9.2. It is functional (and object-oriented), so you can already use it to write and run tests in Scala. It is also quite thoroughly documented. Download and unzip scalatest-0.9.2.zip, and you'll get a scalatest-0.9.2 directory that contains all the good stuff.
If you don't have it already, you'll also need Scala 2.7.1.final, which you can get from here:
ScalaTest is a testing API that also includes a Runner application. You can use the API to create tests and the application
to run them. The ScalaTest API is in package org.scalatest and subpackages.
The main concepts in this API are represented by these three types:
org.scalatest.Suite - a trait whose instances represent test suites (one to many tests)org.scalatest.Reporter - a trait implemented by objects that collect test results and present them to the userorg.scalatest.tools.Runner - a Scala application that runs test suites
ScalaTest provides a variety of traits that facilitate different styles of writing tests.
The most straightforward way to create a test suite is to subclass org.scalatest.Suite (Suite) and define test methods.
A test method is any public method with a name that starts with "test".
A Suite can hold references to other, "nested" Suites.
You organize a large suite of tests by building a tree of Suites. The base Suite in the
tree has nested Suites, each of which may have
nested Suites, and so on. When you execute the base Suite, it makes sure all Suites
in the tree are executed.
For more detail, please check out the ScalaDoc documentation for Suite. This will
give you the run down on how to use a Suite. After that, take a look at the ScalaDoc documentation for
Runner. This will explain the command line used to run tests.
If you want the big picture, you can also:
To try it out, you can use ScalaTest to run some of its own tests, i.e., tests used to test ScalaTest itself. This command will run and just print results to the standard output:
scala -classpath scalatest-0.9.2.jar org.scalatest.tools.Runner -p "scalatest-0.9.2-tests.jar" -o -s org.scalatest.SuiteSuite
This command will run the GUI:
scala -classpath scalatest-0.9.2.jar org.scalatest.tools.Runner -p "scalatest-0.9.2-tests.jar" -g -s org.scalatest.SuiteSuite
You should see something that looks like this:
ScalaTest screenshot
If you want to run the suites concurrently, add a -c parameter:
scala -classpath scalatest-0.9.2.jar org.scalatest.tools.Runner -p "scalatest-0.9.2-tests.jar" -g -c -s org.scalatest.SuiteSuite
ScalaTest has been tested with Scala version 2.7.1.final.
FunSuite and FunSuite1 through FunSuite9, which allow you to express tests as function values.
The “Fun ”in FunSuite stands for functional.
Here's an example FunSuite:
import org.scalatest.fun.FunSuite
class MySuite extends FunSuite {
test("addition") {
val sum = 1 + 1
assert(sum === 2)
assert(sum + 2 === 4)
}
test("subtraction") {
val diff = 4 - 1
assert(diff === 3)
assert(diff - 2 === 1)
}
}
“test” is a method defined in FunSuite, which will be invoked
by the primary constructor of MySuite. You specify the name of the test as
a string between the parentheses, and the test code itself between curly braces.
The test code is a function passed as a by-name parameter to test, which registers
it for later execution. One benefit of FunSuite compared to Suite is you need not name all your
tests starting with “test.” In addition, you can more easily give long names to
your tests, because you need not encode them in camel case, as you must do
with test methods.
FunSuite1 through FunSuite9 facilitate the sharing of mutable fixture objects among tests. If you need
three mutable fixture objects, for example, you'd subclass FunSuite3.
You will need to parameterize FunSuite3 with the types of each
of the fixture objects. Most often you'll do this explicitly in your subclass's declaration. Here's an example:
class MySuite extends FunSuite3[Float, Int, Long] {
testWithFixture("example test") {
(f, i, x) => {
// test code that uses the passed fixture objects...
}
}
def withFixture(f: (Float, Int, Long) => Unit) {
// Create the fixture objects (as in JUnit 3's setUp method)
val f = 1.1f
val i = 7
val x = 21L
// Pass the fixture objects to the test function
f(f, i, x)
// If need be, perform any cleanup (as in JUnit 3's tearDown method)
}
}
This is a contrived example, because normally you would only pass fixture objects in this manner if
they are mutable. (The reason we used immutable objects in this example was just to make the type names fit
more easily on the page.) If they aren't mutable, then the fixture objects can simply be referenced from instance
variables and shared by all test methods that need them. Since they aren't mutable, they can't be "corrupted"
by one test and rendered unusable by the next. For mutable fixture objects, though, this trait allows you
to reinitialize and pass in a fresh set of fixture objects to each test function that needs them. It is essentially
a functional alternative to the setUp and tearDown approach of JUnit and TestNG, which require
reasssigning instance variables between tests.
JUnit3Suite, which allows you to write tests that
that can be run with either JUnit 3 or ScalaTest. JUnit3Suite allows you to write JUnit 3 tests
with ScalaTest's more concise assertion syntax as well as JUnit's assertions (assertEquals, etc.).
You create tests in JUnit 3 fashion, by defining methods that start with test, and can create fixtures with methods
named setUp and tearDown. For example:
import org.scalatest.junit.JUnit3Suite
import scala.collection.mutable.ListBuffer
class TwoSuite extends JUnit3Suite {
var sb: StringBuilder = _
var lb: ListBuffer[String] = _
override def setUp() {
sb = new StringBuilder("ScalaTest is ")
lb = new ListBuffer[String]
}
def testEasy() {
sb.append("easy!")
assert(sb.toString === "ScalaTest is easy!")
assert(lb.isEmpty)
lb += "sweet"
}
def testFun() {
sb.append("fun!")
assert(sb.toString === "ScalaTest is fun!")
assert(lb.isEmpty)
}
}
In this version of ScalaTest, the JUnit integration is in its early stages. JUnit3Suite isn't yet complete, but can still
be used for the most typical use cases. In a future release of ScalaTest, we will add support for groups, nested suites, and concurrent
execution to JUnit3Suite and also provide a JUnit3WrapperSuite that will
allow you to run existing JUnit 3 tests written
in Java with ScalaTest. We also plan to provide a JUnit4Suite and JUnit4WrapperSuite that provides similar integration
for JUnit 4.
To execute JUnitSuites with ScalaTest's Runner, you must include JUnit3's jar file on the class path or runpath.
This version of JUnitSuite was tested with JUnit version 3.8.1.
ScalaTest also provides trait TestNGSuite, which allows you to write tests that
can be run with either TestNG or ScalaTest. TestNGSuite allows you to mark any
method as a test using TestNG's @Test annotation, and supports all other TestNG annotations.
Here's an example:
import org.scalatest.testng.TestNGSuite
import org.testng.annotations.Test
import org.testng.annotations.Configuration
import scala.collection.mutable.ListBuffer
class MySuite extends TestNGSuite {
var sb: StringBuilder = _
var lb: ListBuffer[String] = _
@Configuration { val beforeTestMethod = true }
def setUpFixture() {
sb = new StringBuilder("ScalaTest is ")
lb = new ListBuffer[String]
}
@Test { val invocationCount = 3 }
def easyTest() {
sb.append("easy!")
assert(sb.toString === "ScalaTest is easy!")
assert(lb.isEmpty)
lb += "sweet"
}
@Test { val groups = Array("com.mycompany.groups.SlowTest") }
def funTest() {
sb.append("fun!")
assert(sb.toString === "ScalaTest is fun!")
assert(lb.isEmpty)
}
}
In addition, ScalaTest provides TestNGWrapperSuite, which
wraps existing TestNG test suites described by TestNG XML files. TestNGWrapperSuite allows
existing TestNG tests written in Java to be run by ScalaTest.
One way to use TestNGWrapperSuite is to extend it and specify a list of one or more
names of TestNG XML suite file names to run. Here's an example:
class MyWrapperSuite extends TestNGWrapperSuite(
List("oneTest.xml", "twoTest.xml", "redTest.xml", "blueTest.xml")
)
You can also specify TestNG XML files on Runner's command line with -t parameters. See
the documentation for Runner for more information.
To execute TestNGSuites or TestNGWrapperSuites with ScalaTest's Runner, you must include TestNG's jar file on the class path or runpath.
This version of TestNGSuite was tested with TestNG version 5.7.
ScalaTest provides trait CheckSuite, which contains several “check” methods that can be used to perform ScalaCheck property checks.
A ScalaCheck property check will be reported as ScalaTest test failure if ScalaCheck finds a test case for which the property doesn't hold.
To use ScalaCheck, you specify properties and, in some cases, generators that generate test data. You need not always
create generators because ScalaCheck provides many default generators for you, which can be used in many situations.
ScalaCheck will use the generators to generate test data and with that data run tests that check that the property holds.
Property-based tests can, therefore, give you a lot more testing for a lot less code than asertion-based tests.
Here's an example of using ScalaCheck from a ScalaTest suite:
import org.scalatest.scalacheck.CheckSuite
import org.scalacheck.Arbitrary._
import org.scalacheck.Prop._
class MySuite extends CheckSuite {
def testConcat() {
checkProperty(
(a: List[Int], b: List[Int]) => {
a.size + b.size == (a ::: b).size
}
)
}
}
The checkProperty method, defined in CheckSuite, makes it easy to write property-based tests inside
ScalaTest, JUnit, and TestNG test suites. This example specifies a property that List's ::: method
should obey. ScalaCheck properties are expressed as function values that take as parameters the required
test data, which will be generated by ScalaCheck. In this case, the test data is composed of lists of integer named a and b.
Inside the body of the function, you see:
a.size + b.size == (a ::: b).size
The property in this case is a Boolean expression that will yield true if the size of the concatenated list is equal
to the size of each individual list added together. With this small amount
of code, ScalaCheck will generate possibly hundreds of values for a and b and test each one, looking for
a value for which the property doesn't hold. If the property holds true for every value ScalaCheck tries,
checkProperty returns normally. Otherwise, checkProperty will complete abruptly with an AssertionError that
contains information including the values for a and b that caused the failure.
To execute CheckSuites with ScalaTest's Runner, you must include ScalaCheck's jar file on the class path or runpath.
This version of CheckSuite was tested with ScalaCheck version 1.1.1.
ScalaTest provides an ant task to more conveniently run ScalaTest from Ant.
To use it, you must first define the task in your ant file using taskdef:
<path id="scalatest.classpath">
<pathelement location="${lib}/scalatest.jar"/>
<pathelement location="${lib}/scala-library-2.6.1-final.jar"/>
</path>
<target name="main" depends="dist">
<taskdef name="scalatest" classname="org.scalatest.tools.ScalaTestTask">
<classpath refid="scalatest.classpath"/>
</taskdef>
<scalatest ...
</target>
You can specify user-defined properties using nested <property> elements:
<scalatest>
<property name="dbname" value="testdb"/>
<property name="server" value="192.168.1.188"/>
You can specify a runpath using either a runpath attribute and/or nested
<runpath> elements, using standard ant path notation:
<scalatest runpath="serviceuitest-1.1beta4.jar:myjini">or
<scalatest>
<runpath>
<pathelement location="serviceuitest-1.1beta4.jar"/>
<pathelement location="myjini"/>
</runpath>
To add a URL to your runpath, use a <runpathurl> element
(since ant paths don't support URLs):
<scalatest>
<runpathurl url="http://foo.com/bar.jar"/>
You can specify reporters using nested <reporter> elements, where the type
attribute must be one of the following:
graphic file stdout stderr reporterclass
Each may include a config attribute to specify the reporter configuration.
Types file and reporterclass require additional attributes filename
and classname, respectively:
<scalatest>
<reporter type="stdout" config="FAB"/>
<reporter type="file" filename="test.out"/>
<reporter type="reporterclass" classname="my.ReporterClass"/>
You can specify group includes and excludes using <includes> and
<excludes> elements:
<scalatest>
<includes>
CheckinTests
FunctionalTests
</includes>
<excludes>
SlowTests
NetworkTests
</excludes>
You can specify suites using either a suite attribute or nested
<suite> elements:
<scalatest suite="com.artima.serviceuitest.ServiceUITestkit">
or
<scalatest>
<suite classname="com.artima.serviceuitest.ServiceUITestkit"/>
To specify suites using members-only or wildcard package names, use
either the membersonly or wildcard attributes, or nested
<membersonly> or <wildcard> elements:
<scalatest membersonly="com.artima.serviceuitest">
or
<scalatest wildcard="com.artima.joker">
or
<scalatest>
<membersonly package="com.artima.serviceuitest"/>
<wildcard package="com.artima.joker"/>
Runner from org.scalatest to org.scalatest.tools package.org.scalatest.junit.org.scalatest.testng.org.scalatest.scalacheck.FunSuite family of traits in package org.scalatest.fun.testGroups method name to groups.===, from expected === actual to left === right.2 === 1, from "Expected 1, but got 2" to "2 did not equal 1".expect method name to intercept.expectNPE and expectIAE methods.expect method, which takes an expected and actual value. expect(1) {2} would generate
the error message "Expected 1, but got 2.expect and intercept that take a message.Suite discovery mechanism in Runner, accessed via -m or -w
command-line arguments.!== method from the great Equalizer.ScalaTest 0.9.2 contains many features, but being feature-holics we'd like add a few more for the 1.0 release:
After the 1.0 release, we'd like to attempt an integration with Eric Torreborre's specs that would allow specs assertions to be easily used in ScalaTest suites, and we'd like to integrate with popular Java mocking frameworks.
ScalaTest was written by Bill Venners, Josh Cough, and George Berger, starting in late 2007. ScalaTest, which is almost exclusively written in Scala, follows and improves upon the Java code and design of SuiteRunner, a testing tool also written primarily by Bill Venners, starting in 2001. Over the years a few other people have contributed to SuiteRunner as well, including:
Our goal is to have ScalaTest released 1.0 by the first week of May, 2008, just in time for JavaOne. Please submit feedback in the ScalaTest Forum.
We're in the process of creating a java.net project for scalatest. You can now get the source code from SVN here:
https://scalatest.dev.java.net/source/browse/scalatest/
Thanks, and enjoy.