Writing Unit Tests with Spock Framework: Test Fixtures, Assertions, and Reporting
In this Complete Beginners Guide on Spock, a brief Introduction to Spock Framework and Groovy programming was given in our previous tutorial.
In this tutorial, we will walk through all the details/steps required to get started with Unit testing in Spock.
For the sake of simplicity, we are going to test a simple Calculator Application that has different methods like addition, subtraction, multiplication, division, etc., all of which accept integer parameters and return an integer output.
Table of Contents:
Unit Testing with Spock Video Tutorial
Getting Started
Similar to any other Unit testing framework, Spock too can be used to write scenarios/test cases for an application under test. We’ll try to compare and contrast the different features of the Spock framework with the existing/known frameworks like JUnit.
“def” Keyword
Let us first try to understand Groovy’s “def” keyword in brief. The def keyword is used to define the type-def and can be used to declare a function as well as a field/variable.
“def” is generally used when we don’t want to restrict the type of a field or return type of a method. Let’s see some examples of def keyword in a groovy class and all its valid usages.
// def as variable types
def inputNum = 100
def inputStr = "hello world!!"
def app = new CalculatorApp()
// def as return type of function
def "test function"() {
// function body here
} The Lifecycle of a Spock Specification
Spock spec when executed looks for all the tests defined and executes them one by one. However, there are a few other functionalities/features that are provided by Spock to make tests less redundant and more readable.
Let’s discuss some features below:
Defining Inputs/Variables as a part of Spec
Consider having multiple tests all using the same input values. One way would be to initialize the input values in each test individually, else we can directly define the fields at the spec level and ensure that before each test, the fields will be initialized and available to the test being executed.
Let’s see an example for our calculator application class.
We will define the input data at the spec level so that it will be available with the initial values to all the tests present in the specification.
class CalculatorAppSpec extends Specification {
def input1 = 50
def input2 = 10
def result = 0
def app = new CalculatorApp()
def "addition with valid inputs return expected result"() {
when:
result = app.add(input1, input2)
then:
result == 60
}
def "multiplication with valid inputs return expected result"() {
when:
result = app.multiply(input1, input2)
then:
result == 500
}
def "division with valid inputs return expected result"() {
when:
result = app.divide(input1, input2)
then:
result == 5
}
def "subsctraction with valid inputs return expected result"() {
when:
result = app.substract(input1, input2)
then:
result == 40
}
} In this code sample, you can see, that we have defined input1, input2, the application under test, and the result at the spec level. What this ensures is that every time a test is run from the spec files, the initialized fields are passed to that test. This indeed eliminates the need for setting up tests every time with input values.
Test Fixtures
Similar to most of the unit testing frameworks, Spock also provides setup and cleanup methods for executing special logic/tasks at specific lifecycle events of test execution.
setupSpec & cleanupSpec
These methods are called once for each Spec execution and are called before and after the test execution respectively. These are comparable to @BeforeClass and @AfterClass annotations of JUnit.
setup & cleanup
These methods are called before and after the execution of each test in the spec.
These hooks are the right place for any logic/piece of code that you would like to execute before and after test execution. For Example, In cleanup, you can write a code to close the database connection (if any) that was used during the test.
These can be compared to @BeforeTest and @AfterTest annotations in the JUnit.
Let’s see an example of these fixtures in our calculator application test.
def setupSpec() {
println("###in setup spec!")
}
def cleanupSpec() {
println("###in cleanup spec!")
}
def setup() {
println(">>>in test setup!")
}
def cleanup() {
println(">>>in test cleanup!")
}
If the above test fixture code is added to a spec containing 4 tests, then the output will be as below:
###in setup spec! >>>in test setup! >>>in test cleanup! >>>in test setup! >>>in test cleanup! >>>in test setup! >>>in test cleanup! >>>in test setup! >>>in test cleanup! ###in cleanup spec!
Spock Assertions
The assertions in Spock are called power assert (and it was adopted by Groovy later after being introduced by Spock). The Spock assertions provide a lot of diagnostic exceptions in case of any assert failures.
One can easily find out what went wrong by simply looking at the failure diagnostic as opposed to verbose AssertionErrors in JUnit and other frameworks.
Let’s try understanding this with an Example and contrast it with JUnit
We will work with a simple test that checks for string equality and see what diagnostics are generated in case of an assertion failure.
Spock Test
def "check case-insensitive equality of 2 strings"() {
given: "two input strings"
String str1 = "hello"
String str2 = "HELLO world"
when: "strings are lowercased"
str1 = str1.toLowerCase()
str2 = str2.toLowerCase()
then: "equal strings should return success"
str1 == str2
}
JUnit Test
@Test
public void compareStrings_withValidInput_shouldReturnSuccess() {
// Arrange
String str1 = "hello";
String str2 = "HELLO world";
// Act
str1 = str1.toLowerCase();
str2 = str2.toLowerCase();
// Assert
Assert.assertEquals(str1,str2);
} Spock Output
Condition not satisfied:
str1 == str2
| | |
hello| hello world
false
6 differences (45% similarity)
hello(------)
hello( world)
Expected :hello world
Actual :hello
JUnit Output
org.junit.ComparisonFailure: Expected :hello Actual :hello world
As you can infer from above, the diagnostic info provided by Spock has better detail and is more user-friendly when compared to other frameworks like JUnit.
Assertion Tips and Tricks
Asserting multiple elements at once – Spock provides various shorthands for assertions and one such is “*” notation which allows asserting the elements on the list.
Let’s understand this with an example:
Consider a CityInfo class having city name and population as the fields. We will write a Spock test to assert the names of cities that are there in the given list.
public class CityInfo {
public CityInfo(String cityName, int population) {
this.cityName = cityName;
this.population = population;
}
public String cityName;
public int population;
}
Let’s see the test now:
def "Assert multiple elements of list" () {
given:
def cityList = new LinkedList<CityInfo>()
cityList.add(new CityInfo("Mumbai", 120))
cityList.add(new CityInfo("Delhi", 80))
cityList.add(new CityInfo("Chennai", 100))
expect:
cityList*.cityName == ["Mumbai", "Delhi", "Chennai"]
}
As shown in the assertion shorthand above you could validate the entire list with the help of the “*” keyword.
Let’s also see what a failure would’ve looked like. I’ll remove the name of anyone city from the assertion above.
Condition not satisfied: cityList*.cityName == ["Delhi", "Chennai"] | | | | | false | [Mumbai, Delhi, Chennai] [app.CityInfo@31368b99, app.CityInfo@1725dc0f, app.CityInfo@3911c2a7]
You can see that the diagnostic information of assertion failure is rich and easy to comprehend.
Leveraging closure parameter – every().
Let’s see, how we can leverage the closure parameter named every() to add an assertion for every element of a list or collection. In the same example, let’s try adding an assertion that validates the population of each city if the given input is > 50.
def "Assert multiple elements of list" () {
given:
def cityList = new LinkedList<CityInfo>()
cityList.add(new CityInfo("Mumbai", 120))
cityList.add(new CityInfo("Delhi", 80))
cityList.add(new CityInfo("Chennai", 100))
expect:
cityList*.cityName == ["Mumbai", "Delhi", "Chennai"]
and:
cityList.population.every() {
it > 50
}
}
Asserting Thrown Exceptions
Exceptions can be asserted to be thrown in the “then” block (which means when the block is also required). The exception detail can be diagnosed by assigning the thrown exception to a field and asserting the required properties of the exception thrown.
Let’s use the same CityInfo class and define a method that throws an exception and write a test for it.
public class CityInfo {
public CityInfo(String cityName, int population) {
this.cityName = cityName;
this.population = population;
}
public String cityName;
public int population;
public CityInfo() {
}
public int getCleanlinessScore() {
throw new RuntimeException("method not implemented");
}
}
Let’s look at the test now:
def "cleanliness score throws runtime exception with message - method not implemented"() {
given:
CityInfo app = new CityInfo();
when:
app.cleanlinessScore()
then:
def e = thrown(RuntimeException)
e.message == "method not implemented"
}
Reporting
In order to generate beautiful and detailed HTML-based reports, there are libraries available that can be added to the build file, and now whenever the tests get executed during the build (or by direct execution), a detailed html based report will get generated in the output folder.
In order to get the test reports generated, add the following libraries to the build.gradle file (and similarly for Maven pom.xml file as well).
testCompile 'com.athaydes:spock-reports:1.6.1' testCompile 'org.slf4j:slf4j-api:1.7.13' testCompile 'org.slf4j:slf4j-simple:1.7.13'
Now build the project and execute the tests by running all the tests in the “test” folder or by executing the “gradle clean test”.
You can open the index.html file to get a summarized report for all Spock specs that were available to be executed.
If you want to see the detailed report for a specific Spec, then click on the spec from the above list and you can see a detailed report of failures as well as successes.
Conclusion
In this tutorial, we covered the basics of Unit testing with Spock Framework. We saw the different ways and shorthands for writing assertions and the kind of rich diagnostics info generated by the Spock framework for assertion failures.
We also looked at how we could generate quite pretty HTML-based reports for the Spock tests which include the same detailed diagnostics for the tests executed.
Our upcoming tutorial will brief you about writing parameterized tests with Spock in detail!!