MyFunSuite takes a parameter and constructs its testcases based on the value of the parameter.
Then I have a Suite which contains two (or more) instances of MyFunSuite as nested suites:
class AllTests extends Sequential(new MyFunSuite((1 until 10), "Range 1") , new MyFunSuite((30 until 35), "Range 2"))
When I execute AllTests, e.g. with sbt (> test-only tests.error.AllTests), sbt reports successful runs: [info] Run completed in 383 milliseconds. [info] Total number of tests run: 14 [info] Suites: completed 3, aborted 0 [info] Tests: succeeded 14, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed.
But when I look for the XML-output, I am missing reports on testcases. First of all, I only see an XML-file for the second instance of MyFunSuite but not for the first. I guess it gets overwritten(?) as the filename for the report seems to be per class not per instance. Furthermore the test report for AllTests says it does not contain any tests at all: <testsuite errors="0" failures="0" hostname="xubuntu" name="tests.error.AllTests" tests="0" time="0.148" timestamp="2014-06-20T14:52:57" ...
I would have expected the testcases of the nested suites to count towards the number of tests of the complete suite. The problem with the behaviour as it is is that this confuses Jenkins as it thinks that I do not have any testcases.
Am I making a conceptual error here or is this a potential bug?
For completeness, my build.sbt definition is: name := "TestsuiteError"
You'll need to override suiteId and provide a unique suiteId for each nested suite. By default the suiteId is the fully qualified name of the class, and in this case you have two instances of that class. The XML reporter will write a file for both nested suites, but the second one will overwrite the first one, because it has the same name. This should work:
package com.example
import org.scalatest._
class MyFunSuite(val range: Range, name: String) extends FunSuite {