Risk Management at Test Execution Phase Explained with Practical Example (Part 2)

In our journey to understanding the risk management process, we talked about how it goes exclusively in the => Test planning phase of risk based testing (part 1 tutorial). We also understood the generic process that involves – Risk identification, Risk Assessment and Risk Mitigation Plan.

How to Manage Risk at Test designing or Test execution phase:

There is one other form of Risk management (also sometimes referred to as, Risk based testing) that occurs during the Test designing or Test execution phase depending on the situation. Now, what situation are we talking about? Let us try to understand that first.

We all know that our testing work is reactive. No requirements (or scope defined), we can’t perform feasibility analysis and write test scenarios or plan for testing activities. Similarly, when the code is not ready we have nothing to test, no matter how much of prep-work we might have been ready with in terms of test cases, test data etc. Also, testing is the only step left before the product goes live.

Risk Management – with a focus on the AUT

Risk based testing part 2

Let us understand this better with an example:

If testing was to begin on a said date, Jan 1st and had to go on until Jan 14th – when the testing is done, the product’s Go-live date is usually fixed immediately. Let’s say- Jan 15th for simplicity’s sake. Now, in a perfect world things would go exactly as planned. But we all know the reality.

In this case, let us assume that for some reason, the testing did not start until Jan 7th, which means we lost half of our testing time. But we do need 14 days to test the product thoroughly. We could move the go-live date farther by 7 days- however; this usually is not an option. Because, the product is promised to hit the market on a certain date and any delays are not good for the business. So, usually we testing teams have to absorb the delays, compensate somehow, work with the time available and make sure the product is tested well. Tough job, isn’t it?

This is where the risk management process is again employed.

  • Now, if delays are anticipated ahead of time, before even the testing begins- the process takes place in the Test designing phase.
  • If delays happen during a test execution phase that started normally- the process is followed in the test execution phase.
  • The steps and the method are the same no matter what stage it happens.

What is the process?

Risk management takes place to determine what areas of the AUT (application under test) need maximum focus. These are typically the functional areas (modules or components) that are critical for success of the final product and are most prone to failure.

Read also => Failure mode and effects analysis (FMEA) is a risk management technique

Who performs it?

Since, it is regarding the AUT, the knowledge about it is not only with the QA but with all the other teams – Dev, BA, Client, Project management teams etc. Therefore it is a collective effort, driven by the testing team.

How does Risk Bases Testing take place?

Step #1: Risk Identification

Identify all the functional areas of the AUT. This will simply include making a list.

Step #2: Risk Assessment

All the risks are quantified and prioritized in this step. Quantifying is simply, assigning a number to each risk in the list that will give an indication of the priority with which it needs to be addressed.

Every risk’s probability (the chance of occurrence) and impact (amount of loss that it would cause when this risk materializes) are decided.

The typical method is to assign the ratings. For example, Probability can take values 1 to 5. 1 being probability of occurrence being low (not likely to happen at all) and 5 being high (will most certainly happen).

Similarly, Impact can also be assigned 1-5 rating. 1 being low impact (even if this risk does materialize, the loss is minimum) and 5 being high impact (huge losses when happens).

Step #3:

Make a table format and circulate to all the team reps- Dev, BA, Client, PM, QA and anyone else relevant.

Risk based testing 1

------------

Step #4:

Instruct each team to fill in the values based on their rating for probability and impact.

Since the probability and impact values are numeric it will make the calculating of the “Risk Factor” value easy.

Risk factor = Probability X Impact. The higher the risk factor, the serious the problem is.

An example:

Risk based testing 2

Please note that at this point, this is simply the outcome of one team’s rating. For a project where 5 different teams are involved the QA team would end up with 5 different tables.

Step #5:

Take an average of the Risk factor values. For example, if there are 5 teams, for each module, add all the risk factor values and divide it by 5. This is the final values we are going to deal with. Say, these are the averaged risk factors:

Risk based testing 3

The more the risk factor, the more that module has to be tested.

So, Module 5 and 2 are most crucial for the success of the business. Share the results with the all the teams.

Step #6:

Risk Mitigation Plan – Now, this is the step that changes from Project to Project. We have identified that modules 5 and 2 are the ones that need to be concentrated on most.

Examples of the plan could be:

  1. Modules 5 and 2 will be tested thoroughly by making sure all the test cases related to them are tested. The other modules will be tested on an exploratory basis.
  2. Modules 5 and 2 are going to be tested first and then depending on the time available the others are going to be taken care of.

Once a plan is made, all the teams reach an agreement and follow it to test the product keeping into account the risk-factor.

That’s it!

A few important points to note about Risk Based Testing:

  • Since, this is a collective activity that takes everyone’s opinion into consideration; the chances of it being accurate and effective are higher.
  • This is not a formal method and does not have to be a part of every QA project.
  • Sometimes, even if the team chooses not to draw tables and assign values- a simple brainstorming session with every one present can give the QA team good direction on how to proceed.
  • The development team’s input is very important because they are the ones who create the product, so they will know what might work and what might need additional checking. Be sure to be on the lookout for that.
  • Even though there are multiple steps in the process, it does not take a considerable amount of time to perform them. Especially, if all the teams can sit together and work simultaneously.
  • Remember this process and its outcome is only the alternative. Getting as much time as planned for testing is the best way to perform the QA activity.

Please let us know your comments and questions on this topic below. Also, have you used this formal process in your projects? If yes, please share your experiences.



The Best Software Testing Training You'll Ever Get!

software testing QA training

8 comments ↓

#1 Vinay on 05.12.14 at 11:04 am

Really its a good article.
Now, I got what is Risk based testing.
We ever tried this process.
But, in once we got a input from Dev team to test a particular module. Based on that, we tested and given a demo.

Thanks.

#2 Rohit on 05.12.14 at 2:12 pm

thanks.
the example is perfect to explain the concept.

#3 Rohit Dhilor on 05.13.14 at 12:41 pm

Nice article.

#4 Hitesh Siroya on 05.13.14 at 1:26 pm

Very nice article, explanation is wonderful and user friendly. I will try in my company and lets see what impact.
Kudos for STH team.

#5 Neeraj Verma on 05.15.14 at 9:10 am

Nice Article…I have also followed many articles on this site.It has been really helpful to me for improment in my testing skills..Thanks a lot …

#6 Neeraj Verma on 05.15.14 at 9:16 am

Please correct me if wrong-
While calculating risk , the best person to decide the ,
Impact of the risk – is the Product Owner/Business Analyst
And
Probability ( chance of occurance ) – is a Developer

#7 Swati Seela on 05.27.14 at 5:55 pm

@Neeraj Verma: Yes, you could say that. However, collective conclusions are better than individual opinions. Always.

#8 Yogeesha on 06.17.14 at 10:52 am

Nice article. Thanks

Leave a Comment