Pairwise Testing: Tools and Technique with Example

In this article, we will see what Pairwise Testing is and how it is an Effective Test Design Technique for Finding Defects. Let’s get started. 

Today, we are going to learn about a “Combinatorial Testing” technique called “Pairwise Testing”. It is also known as “All-Pairs Testing”.

Smart testing is the need of the hour. 90% of the time’s system testing team has to work with tight schedules. So test design techniques should be very effective for maximum test coverage and high defect yield rate.

Pairwise Testing

Definition: What is Pairwise Testing

Pairwise Testing is a test design technique that delivers a hundred percent test coverage.

ISTQB defines All-Pairs Testing (or Pairwise Testing) as a black-box test design technique in which test cases are designed to execute all possible discrete combinations of each pair of input parameters.

The output of a software application depends on many factors e.g. input parameters, state variables and environmental configurations. Techniques like boundary value analysis and equivalence partitioning can be useful to identify the possible values for individual factors. But it is impractical to test all possible combinations of values for all those factors. So instead a subset of combinations is generated to satisfy all factors.

The All-Pairs technique is very helpful for designing tests for applications involving multiple parameters. Tests are designed such that for each pair of input parameters to a system, there are all possible discrete combinations of those parameters. The test suite covers all combinations; therefore it is not exhaustive yet very effective in finding bugs.

Learn how to apply the All-Pairs Testing using this example.

Pairwise Testing Example

Car Ordering Application:

• The car ordering application allows for buying and selling cars. This should support trading in Delhi and Mumbai.
• The application should have registration numbers that may be valid or invalid. It should allow you to trade in the following cars: BMW, Audi, and Mercedes.
• Two types of booking can be done: E-booking and In Store.
• Orders can only be placed during trading hours.

Step #1: Let’s list down the variables involved.

1) Order category
a. Buy
b. Sell

2) Location
a. Delhi
b. Mumbai

3) Car brand
a. BMW
b. Audi
c. Mercedes

4) Registration numbers
a. Valid (5000)
b. Invalid

5) Order type
a. E-Booking
b. In-store

6) Order time
a. Working hours
b. Non-working hours

If we want to test all possible valid combinations:
= 2 X 2 X 3 X 5000 X 2 X 2
= 240000  Valid test case combinations 🙁

There is also an infinite number of invalid combinations.

Step #2: Let’s simplify

– Use a smart representative sample.
– Use groups and boundaries, even when data is non-discrete.
– Reduce Registration Number to Two

1. Valid registration number
2. Invalid registration number

Now let’s calculate the number of possible combinations
= 2 X 2 X 3 X 2 X 2 X 2
= 96

Step #3: Arranging variables and values involved.

When we arrange the variables and values involved, it looks something like this:

Now order the variables so that the one with the most number of values is first and the least is last.

Step #4: Arrange variables to create a test suite

Let’s start by filling in the table column by column. Initially, the table should look something like this. The three values of Product (variable having the highest number of values) should be written two times each (two is the number of values of the next highest variable i.e. Order category).

The Order Category column has two values. That’s how many times we need to insert the values of the first column, Product.

For each set of values in column 1, we put both values of column 2. Repeat the same for column 3.

We have a Buy and Delhi, but wait – there’s no Buy and Mumbai. We have a Sell and Mumbai, but there’s no Sell and Delhi. Let’s swap around the values in the second set in the third column.

This looks much better!

We will repeat the same steps for column 3 and 4.

When columns 3 and 4 are compared, each value in column 3 has both the values of column 4. But when you compare the 2^nd and 4^th column, we have Buy and Valid & Sell and Invalid .i.e. Buy does not have an “Invalid” and Sell does not have a “Valid”. Hence we need to interchange the last set of values in the 4^th column.

Column 6 (Order time) is problematic. We are missing the Buy/Non-working hours and Sell/Working hours. We can’t fit our missing pairs by swapping around values as we have already swapped all the rows, if we swap now we may miss other possible pairs that are already sorted. So, we add two more test cases that contain these pairs. Hence, the blank rows!

Now we will fill in the empty cells as we desire because the other variable values are purely arbitrary (or Don’t Cares ~).

Hurray! All pairs are in 8 cases, instead of all combinations in 96!

Hence, we saw how efficient All-pairs technique of test design is. There stands a good chance of finding bugs and it is fun and powerful.

The pairwise testing technique has some limitations as well.

• It fails when the values selected for testing are incorrect.
• It fails when highly probable combinations get too little attention.
• It fails when interactions between the variables are not understood well.
  

Pairwise Testing Tools

Tools are available that apply an all-pair testing technique that facilitates us to effectively automate the Test Case Design process by generating a compact set of parameter value choices as the desired Test Cases. Some well-known tools from the industry are:

• PICT – ‘Pairwise Independent Combinatorial Testing, provided by Microsoft Corp.
• IBM FoCuS – ‘Functional Coverage Unified Solution’, provided by IBM.
• ACTS – ‘Advanced Combinatorial Testing System’, provided by NIST, an agency of the US Government.
  
• Hexawise
• Jenny
• Pairwise by Inductive AS
• VPTag free All-Pair Testing Tool

Conclusion

The pairwise testing technique can dramatically reduce the number of combinations to be covered but remains very effective in terms of fault detection. It is indeed a smart test design technique that guarantees a win-win situation for both test effort and test effectiveness.

During the test planning phase of software testing, the Pairwise testing technique should always be taken into consideration. Either we are doing it manually or using any tool to generate test cases, it becomes a necessary component of the test plan because it, in turn, affects Test estimation.

Hope you liked this article! Feel free to post your feedback in the comments section below. You can also post your queries or doubts. We would love to hear your thoughts.