Explore the Ways of Writing Data-driven or Parameterized Tests with the Spock Framework:
In this Free Spock Training Tutorial Series, we explored all about Unit Testing in Spock and Test fixtures, Assertions and Reporting in our previous tutorial.
In this tutorial, we will try to understand what parameterized tests are and how you can leverage the in-built features of Spock to achieve data-driven testing.
Let’s start!!
Watch the Video Tutorial
Table of Contents:
What are Parameterized Tests?
For anyone who has worked with automation/unit tests, data-driven testing is not a new term.
Parameterized tests are nothing but they are any kind of tests that share the same execution logic and differ only in the input data and outcome in some cases.
Example: Suppose you have a Calculator application, in order to test the functionality completely you might want to run your tests against different input set.
Example: Negative values, fractional numbers, normal integers, integers nearing max allowed range, etc. No matter what input values you have, you want to run the same execution logic.
Another good reason to write parameterized tests is that it does not just test a happy path, rather it also tests error path or negative scenarios.
Example: Suppose there is an application that returns whether a given file extension is valid or not. Data-driven tests can quickly enable the developer to execute tests for supported file extensions and any error scenarios or negative input tests.
Now traditionally, you can think of writing or copying over the tests for multiple input values but, that’s not the correct or smart way to achieve this kind of test execution. Moreover, as the number of tests starts increasing in your app, these tests will become difficult to maintain.
Writing Parameterized Tests with Spock
The where: block
The where block in a Spock test, is the block that holds data for the parameterized test. It can optionally contain both input and expected output values. An important point to note about this block is that this should be the last block in a Spock test.
Having said that, it can be combined with all the other blocks like given, when & then but should be the last block.
Let’s look at an Example to understand it better
We will be using a calculator application that takes 2 input parameters and returns the sum of the supplied inputs. We will be writing a parameterized test supplying multiple inputs and expectedOutput values.
def "sample parameterized test"() {
given:
def app = new CalculatorApp()
when:
def resultSum = app.add(input1, input2)
then:
resultSum == expectedResult
where:
input1 |input2 |expectedResult
10 |15 |25
-4 |6 |2
}
In the above code sample you can see the following:
- “where” block which contains the data for the test to run.
- “where” block is the last block of the test.
- “where” is combined with the other blocks i.e. given, when and then.
- Data representation is a special format called data tables which we will look in detail in the upcoming sections of this tutorial.
- Header row of data is essentially the properties/input variables which can be directly used in the test. E.g. Refer to the statement in the “when” block where we’ve directly used input1 and input2 as input parameters without defining them explicitly.
Using Datatables
Let’s try understanding data tables in detail now. Each line of the data-table represent data for an individual scenario (test execution).
By convention i.e. input values are preceded by a single pipe (‘|’) while output values are preceded by double pipe (‘||’). This does not have any logical significance, but it’s convention & it improves readability. Thus, both the examples below hold true.
input1 |input2 |expectedResult 10 |15 |25 -4 |6 |2 input1 |input2 || expectedResult 10 |15 || 25 -4 |6 || 2
The header row, as shown above, has a name for each of the parameters supplied as data to test. It’s important to note here that these parameter names should not clash with any existing local/global variables in the test, else there will be compile-time errors to resolve variable names.
An important point to note while using data-tables is that a minimum of 2 columns is required. If you just have a need of one column, then a blank column with values as underscore character is a workaround like below.
input1 ||_ 10 ||_ -4 ||_
The advantage of this format is simplicity, readability, and extensibility. Adding a new data input is as simple as adding a new row with data values.
Another point to note here is that data tables can be used to hold any type of variables, classes, objects, enums, etc. which make it even more powerful. As groovy is an optionally typed language, if an explicit type is not specified, the variables in the data table imply depending on the type of data supplied.
Let’s see another Example using data tables with a list of strings as input and output as a count of elements in the string.
def "sample parameterized test with list data type"() {
when:
def actualCount = input1.size()
then:
actualCount == expectedCount
where:
input1 ||expectedCount
["hello","world","happy","programming"] ||4
["spock","data-driven","testing"] ||3
}
In the above example, you can notice that we’ve provided input as an array list of Strings and output is the size of this array list. Thus, it gives a lot of flexibility to have input data of different types.
You can also simply mention any expressions that return data of the respective input type and use in data tables directly as well.
Lifecycle of the “where” Block
For tests containing where block and data samples in the form of data tables, each row of data represent one execution of the test method.
For Example, if there are 5 rows of data and the test contains “given” and “when” blocks, then for such data-row the test blocks will get executed once. So, overall, there will be a total of 5 executions of the test method.
Tips & Tricks
Let’s see some tips and tricks for parameterized-tests while working with these data-tables.
#1) Displaying the results of individual row execution separately. As we saw in the lifecycle section, for each row of data there is one execution of the test code. In order to get these rows or results displayed separately for each such row “@Unroll” annotation can be used for such tests.
Let’s try understanding this with an example:
We will be using the same calculator application with 3 sets of input data being supplied to the method under test.
def "sample parameterized test"() {
given:
def app = new CalculatorApp()
when:
def resultSum = app.add(input1, input1)
then:
resultSum == 2 * input1
where:
input1 |input2 |expectedResult
10 |15 |25
-4 |6 |2
-32 |12 |-20
}
Without “@Unroll” annotation, let’s see how does the result look in the terminal (as well as the html based reports). With this kind of output, it becomes difficult to find out what set of input caused the test to fail.
Now let’s see how the test output is reported separately for each row after adding “@Unroll” annotation to the test method (which has data-tables as data input).
#2) Now, let us understand how to add meaningful info to these data-driven tests (instead of some auto appended indexes as in the screenshot above).
We can use placeholders for the input and output properties (as per data-table) and then we can see the values populated in tests names with data from data-tables.
Let’s use the same example and update the test name to get data from the input and expected output as mentioned in the data tables:
@Unroll
def "result of adding #input1 & #input2 should be #expectedResult"() {
given:
def app = new CalculatorApp()
when:
def resultSum = app.add(input1, input1)
then:
resultSum == 2 * input1
where:
input1 |input2 ||expectedResult
10 |15 ||25
-4 |6 ||2
-32 |12 ||-20
}
Now let’s see how does the output look in the terminal and the HTML based reports:
So, as you can see here the data from input and output is now showing along with the test names when they are getting executed. This way it makes troubleshooting and debugging a lot easier as it clearly indicates what input caused the test to fail or misbehave.
Conclusion
In this tutorial, we learned about writing parameterized tests with the Spock framework. We also discussed various features of data tables and how they can be used.
Check out our upcoming tutorial to know how to use Mocks and Stubs with Spock!!