How to Manage Risks During Test Planning Phase – Risk Based Testing (Part 1)

Risk Management During Test Planning – Risk Based Testing:

Life is full of risks, and so is a software project. Anything can go wrong anytime. We are always on our toes to make things right – but what about making sure that nothing goes wrong and that when it does we know exactly what to do? Enter Risk management – this is a portion of a software testing project that prepares us to prevent, understand, find and get over risks.

A risk is simply a problem that is likely to occur and when it does occur, will cause a loss.

The loss could be anything- money, time, effort or a compromise in quality. Loss is never good. No matter how much of a spin we give it, it’s not positive- and never will be. Therefore Risk management is an integral part of software projects to make sure we handle risks and prevent/reduce losses.

how to manage risks during test planning

Risk Based Testing: Since we are a QA community let us stay specific to risks and the process related to it in our QA world exclusively. Risks are assessed and managed roughly in 2 phases of our Software Test life cycle. STLC can be categorized into 3 phases -Test planning, Test designing and Test execution.

The risk management process occurs twice, during:

  1. Test planning
  2. Test case design(end) or sometimes in the test execution phase

It is mandatory in case 1 but with case 2 it is more of a ‘need-basis’ situation.

risk based testing process

Two part article series:

Even though the underlying process is the same, the types of risks addressed in both these areas are completely different. In order to do justice to them individually, we are going to handle them differently as a two part series.

Today’s article is going to be about “Risk Management during test planning”.

Risk Management Process

The generic process for Risk management involves 3 important stages:

  1. Risk identification
  2. Risk impact analysis
  3. Risk mitigation

Risk identification

As it is said, the first step to solving a problem is identifying it. This stage involves making a list of everything that might potentially come up and disrupt the normal flow of events.

The main outcome of this step is: a list of risks.

This risk based testing step is commonly led by the QA lead/Manager/representative. However, the lead alone will not be able to come up the entire list- the entire QA team’s input makes a huge impact. We can say this is a collective activity led by the QA lead.

Also, the risks that are identified during the Test planning phase are more ‘managerial’ in orientation- meaning, we are going to look at anything that might impact the QA project’s schedule, effort, budget, infrastructure changes, etc. The focus here is not the AUT, but the way the QA phase will go on.

Risks During Test Planning – Risk Based Testing Examples

The following is a sample list of risks that might be listed during test planning phase. Please note, that the AUT and its functionality is not the focus here.

#1. Testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.

#2. Not enough resources, resources on boarding too late (process takes around 15 days.)

#3. Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time consuming to resolve.

#4. Scope not defined completely defined

#5. Natural disasters

#6. Non-availability of Independent Test environment and accessibility

#7. Delayed Testing Due To new Issues

At this point, you can choose to be as thorough as you would like, depending on the amount of time available.

------------

Once, all the risks are listed, we progress to Risk assessment/Risk impact analysis.

Risk assessment/Risk impact analysis

Is your Risk Analysis something like this? :)

risk management cartoon

Risk Analysis in software testing: All the risks are quantified and prioritized in this step. Every risk’s probability (the chance of occurrence) and impact (amount of loss that it would cause when this risk materializes) are determined systematically.

High – medium – low, values are assigned to both the probability and impact for each risk. The risks with “high” probability and “High” impact are taken care of first and then the order follows.

Risk impact analysis table:

Following these steps, the risk impact analysis table for the above risks listed would look something like this (all values are hypothetical and only for understanding purposes):

Risk
Probability
Impact
1. Testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.HighHigh
2. Not enough resources, resources on boarding too late (process takes around 15 days.)MediumHigh
3. Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time consuming to resolve.MediumHigh
4. Scope not defined completely definedMediumMedium
5. Natural disastersLowMedium
6. Non-availability of Independent Test environment and accessibilityMediumHigh
7. Delayed Testing Due To new IssuesMediumHigh

Risk Mitigation

The final step in this Risk Based Testing (RBT) process is to find solutions to plan how to handle each one of these situations. These plans can differ from company to company, project to project and even person to person.

Risk Mitigation Techniques:

Here is an example of what the Risk’s table transforms into when this phase is complete:

Risk
Prob.
Impact
Mitigation Plan
SCHEDULE
Testing schedule is tight. If the start of the testing is delayed due to design tasks, the test cannot be extended beyond the UAT scheduled start date.
HighHigh• The testing team can control the preparation tasks (in advance) and the early communication with involved parties.
• Some buffer has been added to the schedule for contingencies, although not as much as best practices advise.
RESOURCES
Not enough resources, resources on boarding too late (process takes around 15 days.
MediumHighHolidays and vacation have been estimated and built into the schedule; deviations from the estimation could derive in delays in the testing.
DEFECTS
Defects are found at a late stage of the cycle or at a late cycle; defects discovered late are most likely be due to unclear specifications and are time consuming to resolve.
MediumHighDefect management plan is in place to ensure prompt communication and fixing of issues.
SCOPE
Scope completely defined
MediumMediumScope is well defined but the changes are in the functionality are not yet finalized or keep on changing.
Natural disastersLowMediumTeams and responsibilities have been spread to two different geographic areas. In a catastrophic event in one of the areas, there will resources in the other areas needed to continue (although at a slower pace) the testing activities.
Non-availability of Independent Test environment and accessibilityMediumHighDue to non availability of the environment, the schedule gets impacted and will lead to delayed start of Test execution.
Delayed Testing Due To new IssuesMediumHighDuring testing, there is a good chance that some “new” defects may be identified and may become an issue that will take time to resolve.
There are defects that can be raised during testing because of unclear document specification. These defects can yield to an issue that will need time to be resolved.
If these issues become showstoppers, it will greatly impact on the overall project schedule.
If new defects are discovered, the defect management and issue management procedures are in place to immediately provide a resolution.

A few points to note:

  1. The sooner risk management starts in a QA project planning phase, the better.
  2. Of all 3 steps, risk identification is the most important. If anything is not listed and considered for further steps, the risk goes unhandled.
  3. Try to find an ideal time frame for this activity. Remember, too much planning leaves too little time for doing.
  4. Also, after the risk management process, if a new situation comes up, the risk management plan can be altered or updated to reflect the most current condition.
  5. Historical data can be very useful for the success of this process.

 

Conclusion:

This brings us to an end of Risk management (Risk Based Testing) in the Test planning phase. Even though the underlying steps and principle are similar, the risk management process is more concentrated towards the AUT when it happens in the test designing/execution phase.

Stay tuned for it in our next article – Risk management in test execution phase.

As always, your comments and questions are most welcome.

About the author: This article series is written by our STH team member Swati S.



The Best Software Testing Training You'll Ever Get!

software testing QA training

16 comments ↓

#1 smita on 05.08.14 at 6:51 am

which is more important risk management during planning or execution?

#2 Suresh on 05.08.14 at 7:21 am

risk management is a continuous process throughout the SDLC and STLC. But it would have more positive impact when done at proper stages.

#3 Zakir on 05.08.14 at 7:57 am

Good post Swati.
Keep up the good work !!!

@smitha

Need to wait and stay tuned to the next article on Risk management in test execution phase which can answer your question :)

#4 kote on 05.08.14 at 12:46 pm

Pardon me for my poor english. Where it says “Holidays and vacation have been estimated” in the example of mitigation, what exactly does it mean? Does it mean considering working on holidays and vacation if the risk occured?

#5 Swati Seela on 05.09.14 at 5:53 pm

@smita: Both are important and really worth the time. RM at Test planning is almost compulsory.

#6 Swati Seela on 05.09.14 at 5:53 pm

@Suresh: A great observation. Thank you for sharing it with us

#7 Swati Seela on 05.09.14 at 5:55 pm

@kote: nope, it means planning has been thorough to include holidays and vacation times – this means schedule variance(if any)will occur only due to unforeseen issues, but not due to slip-ups during planning. Hope that makes sense.

#8 Amruta Wable on 05.10.14 at 9:57 am

Thank you for this post,its really nice and understand to read risk testing,can you please send me some points about SDLC

#9 kote on 05.14.14 at 2:21 pm

@Swati: sorry for not writing back sooner. I’ve been away in a while with limited Internet access. Thank you for the elaboration. I now have a clear idea!

#10 Rajahmix on 05.15.14 at 3:56 pm

This was the wonderful post, it taught me a lot about risk management especially during test planning process.

#11 Rajesh on 05.19.14 at 6:34 pm

Hi,

The article is good. I have few points to add and how do we mitigate them?. Also, I suggest to identify the maximum risks during test planning phase and document them in Test Plan before requirements review is completed.
1. Schedule risk – requirements got delayed and it has subsequent effect on next tasks like development design/coding, testing.
2. Test Environment Risk – Coding is completed and build is ready to be deployed in test areas but test areas are not yet setup.
3. Resources Risk – The resources planned for testing have resigned from organization during the peak testing phase and they carryout lot of functional knowledge.
4. Communication Risk – There is an communication misunderstanding/problem among requirement engineers, development and testing teams.
5. Product Risk – If two simultaneous releases are planned to merge at the end of testing phase. Due to the issues identified in release that affects the other – the product is under RISK if gets released. The management has to take decision on it.

Do let me know your comments on this.

#12 Swati Seela on 05.22.14 at 5:41 pm

@Rajesh: It is a great question. Risk analysis and planning is a preventive activity but it does not mean it will remove the risk altogether. So, all the situations you mentioned are possible in which we have an issue.

Most issues in IT projects are collective decisions. When these issues do come up- it means its time to sit together and come up with a alternate plan or workaround.

Risk management prepares us to an extent, but the rest of it is still up to the forces of nature :)

#13 Srinath on 05.26.14 at 7:09 am

Its great explanation for @ Swati Seela

#14 Ramesh Bestha on 06.06.14 at 6:13 am

Its great explanation of Risk management for Testing

#15 Sarita Bennett on 06.11.14 at 12:58 am

Outstanding posting; thanks Swati! Thanks Rajesh for your comments and points. I too thought of identifying the maximum risk during the initial test plan and modify throughout the SDLC…

#16 sarvesh on 07.05.14 at 5:43 pm

Hi,
very good post on Risk management. Thanks.
Could I request you to add more in Mitigation plan ? I can see risk/problem is again explained in mitigation plan. Mitigation is to resolve the problem if occurred. i.e. Scope not defined, environment not available, changing requirements.

Thanks,

Leave a Comment