sbt test

Synopsis

sbt [query / ] test [testname1 testname2] [ -- options ]

Description

The test task provides a means for compiling and running the tests.

By default, the test task in sbt 2.x:

1. Subproject parallelism. Performs compilation of the relevant subprojects in parallel, specified by the query.
2. Test suite parallelism. Maps discovered test suites, to tasks and executes them in parallel.
3. Incremental test. Runs only the tests that either failed in the previous run, never run, or if sbt detects changes in the test or its dependencies.
4. Cached. The test result is cached machine-wide, and optionally remote cached.

The standard source locations for testing are:

• Scala sources in src/test/scala/
• Java sources in src/test/java/
• Resources for the test classpath in src/test/resources/

The resources may be accessed from tests by using the getResource methods of java.lang.Class or java.lang.ClassLoader.

Test interfaces

sbt defines the common interface for JVM-based test frameworks, allowing automatic test suite discovery and parallel execution. By default sbt integrates with MUnit, ScalaTest, Hedgehog, ScalaCheck, Specs2, Weaver, ZIO Test, and JUnit 4; this means you only need to add the test framework to the classpath to work with sbt. For example, MUnit may be used by declaring it as a libraryDependency:

lazy val munit = "org.scalameta" %% "munit" % "1.1.1"

libraryDependencies += munit % Test

In the above, Test denotes the Test configuration, and means that MUnit will only be on the test classpath and it isn't needed by the main sources.

JUnit

Support for JUnit 5 is provided by sbt-jupiter-interface. To add JUnit Jupiter support into your project, add the jupiter-interface dependency in your project's main build.sbt file.

libraryDependencies += "com.github.sbt.junit" % "jupiter-interface" % "0.15.1" % Test

and the sbt-jupiter-interface plugin to your project/plugins.sbt:

addSbtPlugin("com.github.sbt.junit" % "sbt-jupiter-interface" % "0.15.1")

Support for JUnit 4 is provided by junit-interface. Add the junit-interface dependency in your project's main build.sbt file.

libraryDependencies += "com.github.sbt" % "junit-interface" % "0.13.3" % Test

Test filtering

In sbt 2.x, the test task accepts a whitespace separated list of test names to run. For example:

> test example.ExampleSuite example.ExampleSuite2

Here's an example output:

> test example.ExampleSuite example.ExampleSuite2
[info] compiling 1 Scala source to /tmp/foo/target/out/jvm/scala-3.7.2/foo/backend ...
[info] compiling 2 Scala sources to /tmp/foo/target/out/jvm/scala-3.7.2/foo/test-backend ...
example.ExampleSuite:
  + addition 0.003s
example.ExampleSuite2:
  + subtraction 0.003s
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] elapsed time: 3 s, cache 49%, 25 disk cache hits, 26 onsite tasks

It supports wildcards as well:

> test *Example*

Incremental testing

In addition to the explicit filter, the test task runs only the tests that satisfy one of the following conditions are run:

• The tests that failed in the previous run
• The tests that were not run before
• The tests that have one or more transitive dependencies, maybe in a different project, recompiled.

Full testing

To run, uncached full tests, like sbt 1.x, use the testFull task.

Other tasks

Tasks that are available for main sources are generally available for test sources, but are prefixed with Test / on the command line and are referenced in Scala code with Test / as well. These tasks include:

• Test / compile
• Test / console
• Test / consoleQuick
• Test / run
• Test / runMain

See sbt run for details on these tasks.

Output

By default, logging is buffered for each test source file until all tests for that file complete. This can be disabled by setting logBuffered:

Test / logBuffered := false

Test Reports

By default, sbt will generate JUnit XML test reports for all tests in the build, located in the target/test-reports directory for a project. This can be disabled by disabling the JUnitXmlReportPlugin

val myProject = (project in file(".")).disablePlugins(plugins.JUnitXmlReportPlugin)

Options

Test framework arguments

Arguments to the test framework may be provided on the command line to the test tasks following a -- separator. For example:

> test org.example.MyTest -- -verbosity 1

To specify test framework arguments as part of the build, add options constructed by Tests.Argument:

Test / testOptions += Tests.Argument("-verbosity", "1")

To specify them for a specific test framework only:

Test / testOptions += Tests.Argument(TestFrameworks.ScalaCheck, "-verbosity", "1")

Setup and Cleanup

Specify setup and cleanup actions using Tests.Setup and Tests.Cleanup. These accept either a function of type () => Unit or a function of type ClassLoader => Unit. The variant that accepts a ClassLoader is passed the class loader that is (or was) used for running the tests. It provides access to the test classes as well as the test framework classes.

Examples:

Test / testOptions += Tests.Setup( () => println("Setup") )
Test / testOptions += Tests.Cleanup( () => println("Cleanup") )
Test / testOptions += Tests.Setup( loader => ... )
Test / testOptions += Tests.Cleanup( loader => ... )

Disable parallel execution of test suites

By default, sbt runs all tasks in parallel and within the same JVM as sbt itself. Because each test suite is mapped to a task, tests are also run in parallel by default. To make tests within a given project execute serially:

Test / parallelExecution := false

Note that tests from different projects may still execute concurrently.

Filter classes

If you want to only run test classes whose name ends with "Test", use Tests.Filter:

Test / testOptions := Seq(Tests.Filter(s => s.endsWith("Test")))

Forking tests

The setting:

Test / fork := true

specifies that all tests will be executed in a single external JVM.

More control over how tests are assigned to JVMs and what options to pass to those is available with testGrouping key.

Control the number of forked JVMs allowed to run at the same time by setting the limit on Tags.ForkedTestGroup tag, which is 1 by default. Setup and Cleanup actions cannot be provided with the actual test class loader when a group is forked.