What Is Orthogonal Array Testing Technique (OATS)?

This Tutorial Explains What is Orthogonal Array Testing Technique? Learn the Terminology, Implementation, Advantages & Limitations of OATS in This Guide:

Test teams often face ongoing challenges of how to test an application appropriately within the tight deadlines.

Under such circumstances, Exhaustive Testing proves impractical by creating challenges such as a large number of Test Script to execute, how to prioritize the scripts, human error, and fatigue in case of too many scripts being executed by the same person, etc.

Thus in order to face such challenges, Applied Statistics is used in the process of testing an application. This, in turn, helps to execute a wider range of test scripts without compromising the quality and efficiency of the test.

Orthogonal Array Testing Technique (OATS) Technique

One of the most important Applied Statistics techniques is the Orthogonal Array Testing technique which will be discussed in detail in this article. At the end of this article, the reader will have a clear understanding of the implementation of Orthogonal Array Testing in their own application along with its advantages and application technique.

What Is Orthogonal Array Testing (OATS)?

Orthogonal Array Testing technique is a statistical approach for testing pair-wise interactions. Most of the defects which I have observed are caused due to interaction and integration.

This interaction or integration can be within different objects, elements, options in a screen of the application, or configuration setting in a file. Such a combination of objects and elements results in the functioning of the application.

It is obvious that some of the combinations are missed to test which thereby results in insufficient tests. Hence in order to cover the entire functionality in the testing scope with the correct amount of combinations to be tested, Orthogonal Array Testing is used.

This is a Combinational Test Technique that ensures that the complete functionality of an application is tested with a limited and proportionate amount of combinations under test without compromising on the quality of testing.

The beauty of this technique is that it maximizes the coverage by a comparatively lesser number of test cases. The pairs of parameters which are identified should be independent of each other. It’s a black box technique, so like other BB techniques; we don’t need to have the implementation knowledge of the system. The point here is to identify the correct pair of input parameters. 

There are many techniques of CTD, where the OATS (Orthogonal Array Testing Technique) is widely used.

Terminologies in Orthogonal Array Testing

Before understanding the actual implementation of Orthogonal Array Testing, it is essential to understand the terminologies related to it.

Enlisted below are the widely used terminologies in Orthogonal Array Testing:

TermDescription
RunsIt is the number of rows which represents the number of test conditions to be performed.
FactorsIt is the number of columns which represents in the number of variable to be tested
LevelsIt represents the number of values for a Factor
  • As the rows represent the number of test conditions (experiment test) to be performed, the goal is to minimize the number of rows as much as possible.
  • Factors indicate the number of columns which is the number of variables.
  • Levels represent the maximum number of values for a factor (0 – levels – 1). Together, the values in Levels and Factors are called LRUNS (Levels**Factors).

Also Read => State Transition Testing Technique

Implementation Technique of OATS

The Orthogonal Array Testing technique has the following steps:

#1) Decide the number of variables that will be tested for interaction. Map these variables to the factors of the array.

#2) Decide the maximum number of values that each independent variable will have. Map these values to the levels of the array.

#3) Find a suitable orthogonal array with the smallest number of runs. The number of runs can be derived from various websites. One such website is listed here.

#4) Map the factors and levels onto the array.

After performing the above steps, your Array will be ready for testing with all the possible combinations covered.

Example 1

Let’s say that the pages or links in the Software Testing Help page (www.softwaretestinghelp.com) have three dynamic frames (Sections) that can be made as hidden or visible.

OAT Example

Step 1: Determine the number of independent variables. There are three independent variables (sections on the page) = 3 Factors.

Step 2: Determine the maximum number of values for each variable. There are two values (hidden and visible) = 2 Levels.

Step 3: Determine the Orthogonal Array with 3 Factors and 2 Levels. Referring to the link we have derived the number of rows required i.e. 4 Rows.

Orthogonal array follows the pattern LRuns(LevelsFactors). Hence in this example, the Orthogonal Array will be L4(23).

Orthogonal Array

Thus the Orthogonal Array will look in this way.

RunsFactor 1Factor 2Factor 3
Run 1000
Run 2011
Run 3101
Run 4110

Step 4: Map the Factors and Levels of the Array generated.

  • “0” will be replaced by Hidden.
  • “1” will be replaced by Visible.
  • “Factor 1” will be replaced by Section 1.
  • “Factor 2” will be replaced by Section 2.
  • “Factor 3” will be replaced by Section 3.

After mapping the Factors and Levels, the Orthogonal Array will look as shown below:

RunsSection 1Section 2Section 3
Run 1HiddenHiddenHidden
Run 2HiddenVisibleVisible
Run 3VisibleHiddenVisible
Run 4VisibleVisibleHidden

Step 5: Each Run in the above table represents the test scenario to be covered in testing. Each run is changed to a test condition.

Hence while executing such test conditions, a tester will put the conditions as follows:

  • Display Homepage and hide all sections.
  • Display Homepage and show all Sections except Section 1.
  • Display Homepage and show all Sections except Section 2.
  • Display Homepage and show all Sections except Section 3.

Example 2

We provide our personal information like Name, Age, qualification, etc., in various registration forms like first-time app installation or any other Government websites.

The following example is from such kind of application form. Consider that there are four fields in a registration form (webpage) which are having certain sub-options in it.

Age field

  • Less than 18
  • More than 18
  • More than 60

Gender field

  • Male
  • Female
  • NA

Highest Qualification

  • High School
  • Graduation
  • Post-Graduation

Mother Tongue

  • Hindi
  • English
  • Other

Step 1: Determine the number of independent variables. There are four independent variables (Fields of the registration form) = 4 Factors.

Step 2: Determine the maximum number of values for each variable. There are three values (There are three sub-options under each field) = 3 Levels.

Step 3: Determine the Orthogonal Array with 4 Factors and 3 Levels. Referring to the link we have derived the number of rows required i.e. 9 Rows.

Orthogonal array follows the pattern LRuns(LevelsFactors). Hence in this example, the Orthogonal Array will be L9(34).

Thus the Orthogonal Array will look as given below.

RunsFactor 1Factor 2Factor 3Factor 4
Run 10000
Run 20121
Run 30212
Run 41022
Run 51110
Run 61201
Run 72011
Run 82102
Run 92220

Step no. 4: Map the Factors and Levels of the Array generated.

  • “Factor 1” will be replaced by AGE.
  • “Factor 2” will be replaced by Gender.
  • “Factor 3” will be replaced by Highest Qualification.
  • “Factor 4” will be replaced by Mother Tongue.
  • 0, 1, 2 will be replaced by each sub-option under their respective Factor (field).

After mapping the Factors and Levels, the Orthogonal Array will look as shown below:

RunsAGEGenderHighest QualificationMother Tongue
Run 1Less than 18MaleHigh SchoolHindi
Run 2Less than 18FemalePost-GraduationEnglish
Run 3Less than 18NAGraduationOther
Run 4More than 18MalePost-GraduationOther
Run 5More than 18FemaleGraduationHindi
Run 6More than 18NAHigh SchoolEnglish
Run 7More than 60MaleGraduationEnglish
Run 8More than 60FemaleHigh SchoolOther
Run 9More than 60NAPost-GraduationHindi

Step no. 5: Each Run in the above table represents the test scenario to be covered in testing. Each run is changed to a test condition.

Advantages of Orthogonal Array Testing

This technique is beneficial when we have to test with a huge number of data having many permutations and combinations.

  • Less number of Test conditions which requires less implementation time.
  • Less Execution time.
  • Easy Analysis of Test condition due to less number of Test conditions.
  • High coverage of codes.
  • Increased overall productivity and ensures that the quality test is performed.

Limitations of OATS

None of the testing technique provides a guarantee of 100% coverage. Each technique has its way of selecting the test conditions. On similar lines, there are some limitations to using this technique:

  • Testing will fail if we fail to identify the good pairs.
  • Probability of not identifying the most important combination which can result in losing a defect.
  • This technique will fail if we do not know the interactions between the pairs.
  • Applying only this technique will not ensure complete coverage.
  • It can find only those defects which arise due to pairs, as input parameters.

Conclusion

Orthogonal Array testing is a systematical and statistical way of testing pair-wise interactions. It is done by deriving small sets of test cases from a large number of scenarios and also by giving precedence to factors and levels that appear multiple times in the combinatorial outputs.

We can use Orthogonal Array testing in our day to day application testing by:

  • Forming systematic, statistical pair-wise combinations of factors across their levels.
  • Creating an optimized test suite with fewer test scenarios and generating negative test case optimization.
  • Detecting all single, double, and triple mode defects in the given input combinations.
  • Executing a concise set of tests and uncovering most of the bugs.

Now as you have a clear understanding of the implementation of Orthogonal Array testing, you can easily implement it in your application or webpage which will cover all the aspects of the functionality of the application in a limited number of test cases.

We hope this article enriched your knowledge of the concept of Orthogonal Array Testing!!

PREV Tutorial | NEXT Tutorial