How to Manage Risks During Test Planning Phase – Risk Based Testing (Part 1)

Risk Management During Test Planning – Risk Based Testing:

Life is full of risks, and so is a software project. Anything can go wrong anytime. We are always on our toes to make things right – but what about making sure that nothing goes wrong and that when it does we know exactly what to do? Enter Risk management – this is a portion of a software testing project that prepares us to prevent, understand, find and get over risks.

A risk is simply a problem that is likely to occur and when it does occur, will cause a loss.

The loss could be anything- money, time, effort or a compromise in quality. The loss is never good. No matter how much of a spin we give it, it’s not positive- and never will be. Therefore Risk management is an integral part of software projects to make sure we handle risks and prevent/reduce losses.

Risk-Based Testing: Since we are a QA community let us stay specific to risks and the process related to it in our QA world exclusively. Risks are assessed and managed roughly in 2 phases of our Software Test life cycle. STLC can be categorized into 3 phases -Test planning, Test designing and Test execution.

The risk management process occurs twice, during:

  1. Test planning
  2. Test case design(end) or sometimes in the test execution phase

It is mandatory in case 1 but with case 2 it is more of a ‘need-basis’ situation.

Two part article series:

Even though the underlying process is the same, the types of risks addressed in both these areas are completely different. In order to do justice to them individually, we are going to handle them differently as a two-part series.

Today’s article is going to be about “Risk Management during test planning”.

Risk Management Process

The generic process for Risk management involves 3 important stages:

  1. Risk identification
  2. Risk impact analysis
  3. Risk mitigation

What You Will Learn:

Risk identification

As it is said, the first step to solving a problem is identifying it. This stage involves making a list of everything that might potentially come up and disrupt the normal flow of events.

The main outcome of this step is a list of risks.

This risk-based testing step is commonly led by the QA lead/Manager/representative. However, the lead alone will not be able to come up the entire list- the entire QA team’s input makes a huge impact. We can say this is a collective activity led by the QA lead.

Also, the risks that are identified during the Test planning phase are more ‘managerial’ in orientation- meaning, we are going to look at anything that might impact the QA project’s schedule, effort, budget, infrastructure changes, etc. The focus here is not the AUT, but the way the QA phase will go on.



Risks During Test Planning – Risk-Based Testing Examples

The following is a sample list of risks that might be listed during test planning phase. Please note, that the AUT and its functionality is not the focus here.

#1. The testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.

#2. Not enough resources, resources onboarding too late (process takes around 15 days.)

#3. Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time-consuming to resolve.

#4. Scope not defined completely defined

#5. Natural disasters

#6. Non-availability of Independent Test environment and accessibility

#7. Delayed Testing Due To new Issues

At this point, you can choose to be as thorough as you would like, depending on the amount of time available.

Once, all the risks are listed, we progress to Risk assessment/Risk impact analysis.

Risk assessment/Risk impact analysis

Is your Risk Analysis something like this? :)

Risk Analysis in software testing: All the risks are quantified and prioritized in this step. Every risk’s probability (the chance of occurrence) and impact (amount of loss that it would cause when this risk materializes) are determined systematically.

High – medium – low, values are assigned to both the probability and impact of each risk. The risks with “high” probability and “High” impact are taken care of first and then the order follows.

Risk impact analysis table:

Following these steps, the risk impact analysis table for the above risks listed would look something like this (all values are hypothetical and only for understanding purposes):

RiskProbabilityImpact
1. Testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.HighHigh
2. Not enough resources, resources on boarding too late (process takes around 15 days.)MediumHigh
3. Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time consuming to resolve.MediumHigh
4. Scope not defined completely definedMediumMedium
5. Natural disastersLowMedium
6. Non-availability of Independent Test environment and accessibilityMediumHigh
7. Delayed Testing Due To new IssuesMediumHigh

Risk Mitigation

The final step in this Risk Based Testing (RBT) process is to find solutions to plan how to handle each one of these situations. These plans can differ from company to company, project to project and even person to person.

Risk Mitigation Techniques:

Here is an example of what the Risk’s table transforms into when this phase is complete:

RiskProb.ImpactMitigation Plan
SCHEDULE

Testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.

HighHigh• The testing team can control the preparation tasks (in advance) and the early communication with involved parties.

• Some buffer has been added to the schedule for contingencies, although not as much as best practices advise.

RESOURCES

Not enough resources, resources on boarding too late (process takes around 15 days.

MediumHighHolidays and vacation have been estimated and built into the schedule; deviations from the estimation could derive in delays in the testing.
DEFECTS

Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time consuming to resolve.

MediumHighDefect management plan is in place to ensure prompt communication and fixing of issues.
SCOPE

Scope completely defined

MediumMediumScope is well defined but the changes are in the functionality are not yet finalized or keep on changing.
Natural disastersLowMediumTeams and responsibilities have been spread to two different geographic areas. In a catastrophic event in one of the areas, there will resources in the other areas needed to continue (although at a slower pace) the testing activities.
Non-availability of Independent Test environment and accessibilityMediumHighDue to non availability of the environment, the schedule gets impacted and will lead to delayed start of Test execution.
Delayed Testing Due To new IssuesMediumHighDuring testing, there is a good chance that some “new” defects may be identified and may become an issue that will take time to resolve.

There are defects that can be raised during testing because of unclear document specification. These defects can yield to an issue that will need time to be resolved.

If these issues become showstoppers, it will greatly impact on the overall project schedule.

If new defects are discovered, the defect management and issue management procedures are in place to immediately provide a resolution.

A few points to note:

  1. The sooner risk management starts in a QA project planning phase, the better.
  2. Of all 3 steps, risk identification is the most important. If anything is not listed and considered for further steps, the risk goes unhandled.
  3. Try to find an ideal time frame for this activity. Remember, too much planning leaves too little time for doing.
  4. Also, after the risk management process, if a new situation comes up, the risk management plan can be altered or updated to reflect the most current condition.
  5. Historical data can be very useful for the success of this process.

 

Conclusion:

This brings us to an end of Risk management (Risk Based Testing) in the Test planning phase. Even though the underlying steps and principle are similar, the risk management process is more concentrated towards the AUT when it happens in the test designing/execution phase.

Stay tuned for it in our next article – Risk management in test execution phase.

As always, your comments and questions are most welcome.

About the author: This article series is written by our STH team member Swati S.