What Is Orthogonal Array Testing Technique (OATS)?

This Tutorial Explains The Orthogonal Array Testing Technique. Learn the Terminology, Implementation, Advantages & Limitations of OATS in This Guide:

Test teams often face ongoing challenges of how to test an application appropriately within tight deadlines.

Under such circumstances, Exhaustive Testing proves impractical by creating challenges such as a large number of Test scripts to execute, how to prioritize the scripts, human error, fatigue in case of too many scripts being executed by the same person, etc.

Thus to face such challenges, Applied Statistics is used in the process of testing an application. This, in turn, helps to execute a wider range of test scripts without compromising the quality and efficiency of the test.

One of the most important Applied Statistics techniques is the Orthogonal Array Testing technique which will be discussed in detail in this article. At the end of this article, the reader will have a clear understanding of the implementation of Orthogonal Array Testing in their own application along with its advantages and application technique.

What Is Orthogonal Array Testing (OATS)?

The orthogonal Array Testing technique is a statistical approach for testing pair-wise interactions. Most of the defects which I have observed are caused due to interaction and integration.

This interaction or integration can be within different objects, elements, options on a screen of the application, or configuration settings in a file. Such a combination of objects and elements results in the functioning of the application.

Some of the combinations are missed to test which thereby results in insufficient tests. Hence to cover the entire functionality in the testing scope with the correct amount of combinations to be tested, Orthogonal Array Testing is used.

This is a Combinational Test Technique that ensures that the complete functionality of an application is tested with a limited and proportionate amount of combinations under test without compromising on the quality of testing.

The beauty of this technique is that it maximizes the coverage by a comparatively lesser number of test cases. The pairs of parameters that are identified should be independent of each other. It’s a black box technique, so like other BB techniques; we don’t need to have the implementation knowledge of the system. The point here is to identify the correct pair of input parameters. 

There are many techniques of CTD, where the OATS (Orthogonal Array Testing Technique) is widely used.

Terminologies for Orthogonal Array Testing

Before understanding the actual implementation of Orthogonal Array Testing, it is essential to understand the terminologies related to it.

Enlisted below are the widely used terminologies for Orthogonal Array Testing:

• As the rows represent the number of test conditions (experiment test) to be performed, the goal is to minimize the number of rows as much as possible.
• Factors indicate the number of columns, which is the number of variables.
• Levels represent the maximum number of values for a factor (0 – levels – 1). Together, the values in Levels and Factors are called LRUNS (Levels**Factors).

Also, Read => State Transition Testing Technique

Implementation Techniques of OATS

The Orthogonal Array Testing technique has the following steps:

#1) Decide the number of variables that will be tested for interaction. Map these variables to the factors of the array.

#2) Decide the maximum number of values that each independent variable will have. Map these values to the levels of the array.

#3) Find a suitable orthogonal array with the smallest number of runs. The number of runs can be derived from various websites. One such website is listed here.

#4) Map the factors and levels onto the array.

#5) Translate them into suitable Test Cases

#6) Look out for the leftover or special Test Cases (if any)

After performing the above steps, your Array will be ready for testing with all the possible combinations covered.

Example 1

Let’s say that the pages or links on the Software Testing Help page (www.softwaretestinghelp.com) have three dynamic frames (Sections) that can be made hidden or visible.

Step 1: Determine the number of independent variables. There are three independent variables (sections on the page) = 3 Factors.

Step 2: Determine the maximum number of values for each variable. There are two values (hidden and visible) = 2 Levels.

Step 3: Determine the Orthogonal Array with 3 Factors and 2 Levels. Referring to the link we have derived the number of rows required i.e. 4 Rows.

The orthogonal array follows the pattern L_Runs(Levels^Factors). Hence in this example, the Orthogonal Array will be L4(2³).

Thus the Orthogonal Array will look this way.

Step 4: Map the Factors and Levels of the Array generated.

• “0” will be replaced by Hidden.
• “1” will be replaced by Visible.
• “Factor 1” will be replaced by Section 1.
• “Factor 2” will be replaced by Section 2.
• “Factor 3” will be replaced by Section 3.

After mapping the Factors and Levels, the Orthogonal Array will look as shown below:

Step 5: Each Run in the above table represents the test scenario to be covered in testing. Each run is changed to a test condition.

Hence, while executing such test conditions, the tester will put the following conditions:

• Display the Homepage and hide all sections.
• Display the Homepage and show all Sections except Section 1.
• Display the Homepage and show all Sections except Section 2.
• Display the Homepage and show all Sections except Section 3.

Example 2

We provide our personal information like Name, Age, Qualification, etc., in various registration forms like first-time app installation or any other Government websites.

The following example is from this kind of application form. Consider that there are four fields in a registration form (webpage) which have certain sub-options in it.

Age field

• Less than 18
• More than 18
• More than 60

Gender field

• Male
• Female
• NA

Highest Qualification

• High School
• Graduation
• Post-Graduation

Mother Tongue

• Hindi
• English
• Other

Step 1: Determine the number of independent variables. There are four independent variables (Fields of the registration form) = 4 Factors.

Step 2: Determine the maximum number of values for each variable. There are three values (There are three sub-options under each field) = 3 Levels.

Step 3: Determine the Orthogonal Array with 4 Factors and 3 Levels. Referring to the link we have derived the number of rows required i.e. 9 Rows.

The orthogonal array follows the pattern L_Runs(Levels^Factors). Hence, in this example, the Orthogonal Array will be L9(3⁴).

Thus the Orthogonal Array will look as given below.

Step no. 4: Map the Factors and Levels of the Array generated.

• “Factor 1” will be replaced by AGE.
• “Factor 2” will be replaced by Gender.
• “Factor 3” will be replaced by Highest Qualification.
• “Factor 4” will be replaced by Mother Tongue.
• 0, 1, 2 will be replaced by each sub-option under their respective Factor (field).

After mapping the Factors and Levels, the Orthogonal Array will look as shown below:

Step no. 5: Each Run in the above table represents the test scenario to be covered in testing. Each run is changed to a test condition.

Advantages of Orthogonal Array Testing

This technique is beneficial when we have to test with a huge amount of data having many permutations and combinations.

• Less number of Test conditions that require less implementation time.
• Less Execution time.
• Easy Analysis of Test conditions due to less number of Test conditions.
• High coverage of codes.
• Increased overall productivity and ensures that the quality test is performed.

Limitations of OATS

None of the testing techniques provides a guarantee of 100% coverage. Each technique has its way of selecting the test conditions. Along similar lines, there are some limitations to using this technique:

• Testing will fail if we fail to identify the good pairs.
• Probability of not identifying the most important combination which can result in losing a defect.
• This technique will fail if we do not know the interactions between the pairs.
• Applying only this technique will not ensure complete coverage.
• It can find only those defects which arise due to pairs, as input parameters.

Conclusion

Orthogonal Array testing is a systematical and statistical way of testing pair-wise interactions. It is done by deriving small sets of test cases from a large number of scenarios and also by giving precedence to factors and levels that appear multiple times in the combinatorial outputs.

Orthogonal Array testing can be used in our day-to-day application testing by:

• Forming systematic and statistical pair-wise combinations of factors across their levels.
• Creating an optimized test suite with fewer test scenarios and generating negative test case optimization.
• Detecting all single, double, and triple mode defects in the given input combinations.
• Executing a concise set of tests and uncovering most of the bugs.

Now, as you have a clear understanding of the implementation of Orthogonal Array testing, you can easily implement it in your application or webpage which will cover all the aspects of the functionality of the application in a limited number of test cases.

We hope this article enriches your knowledge of the concept of Orthogonal Array Testing!!

PREV Tutorial | NEXT Tutorial