Risk Management at Test Execution Phase Explained with Practical Example (Part 2)

In our journey to understanding the risk management process, we talked about how it goes exclusively in the => Test planning phase of risk-based testing (part 1 tutorial). We also understood the generic process that involves – Risk identification, Risk Assessment and Risk Mitigation Plan.

How to Manage Risk at Test designing or Test execution phase:

There is one other form of Risk management (also sometimes referred to as, Risk-based testing) that occurs during the Test designing or Test execution phase depending on the situation. Now, what situation are we talking about? Let us try to understand that first.

We all know that our testing work is reactive. No requirements (or scope defined), we can’t perform a feasibility analysis and write test scenarios or plan for testing activities. Similarly, when the code is not ready we have nothing to test, no matter how much of prep-work we might have been ready with in terms of test cases, test data etc. Also, testing is the only step left before the product goes live.

Risk Management – with a focus on the AUT

Let us understand this better with an example:

If testing was to begin on a said date, Jan 1st and had to go on until Jan 14th – when the testing is done, the product’s Go-live date is usually fixed immediately. Let’s say- Jan 15th for simplicity’s sake. Now, in perfect world things would go exactly as planned. But we all know the reality.

In this case, let us assume that for some reason, the testing did not start until Jan 7th, which means we lost half of our testing time. But we do need 14 days to test the product thoroughly. We could move the go-live date farther by 7 days- however; this usually is not an option. Because the product is promised to hit the market on a certain date and any delays are not good for the business. So, usually, we testing teams have to absorb the delays, compensate somehow, work with the time available and make sure the product is tested well. Tough job, isn’t it?

This is where the risk management process is again employed.

What is the process?

Risk management takes place to determine what areas of the AUT (application under test) need maximum focus. These are typically the functional areas (modules or components) that are critical for the success of the final product and are most prone to failure.

Read also => Failure mode and effects analysis (FMEA) is a risk management technique

Who performs it?

Since it is regarding the AUT, the knowledge about it is not only with the QA but with all the other teams – Dev, BA, Client, Project management teams etc. Therefore it is a collective effort, driven by the testing team.

How does Risk Bases Testing take place?

Step #1: Risk Identification

Identify all the functional areas of the AUT. This will simply include making a list.

Step #2: Risk Assessment

All the risks are quantified and prioritized in this step. Quantifying is simply, assigning a number to each risk in the list that will give an indication of the priority with which it needs to be addressed.

Every risk’s probability (the chance of occurrence) and impact (amount of loss that it would cause when this risk materializes) are decided.

The typical method is to assign the ratings. For example, Probability can take values 1 to 5. 1 being the probability of occurrence being low (not likely to happen at all) and 5 being high (will most certainly happen).

Similarly, Impact can also be assigned a 1-5 rating. 1 being low impact (even if this risk does materialize, the loss is minimum) and 5 being high impact (huge losses when happens).

Step #3:

Make a table format and circulate to all the team reps- Dev, BA, Client, PM, QA and anyone else relevant.

Step #4:

Instruct each team to fill in the values based on their rating for probability and impact.

Since the probability and impact values are numeric it will make the calculating of the “Risk Factor” value easy.

Risk factor = Probability X Impact. The higher the risk factor, the serious the problem is.

An example:

Please note that at this point, this is simply the outcome of one team’s rating. For a project where 5 different teams are involved the QA team would end up with 5 different tables.

Step #5:

Take an average of the Risk factor values. For example, if there are 5 teams, for each module, add all the risk factor values and divide it by 5. This is the final values we are going to deal with. Say, these are the averaged risk factors:

The more the risk factor, the more that module has to be tested.

So, Module 5 and 2 are most crucial for the success of the business. Share the results with the all the teams.

Step #6:

Risk Mitigation Plan – Now, this is the step that changes from Project to Project. We have identified that modules 5 and 2 are the ones that need to be concentrated on most.

Examples of the plan could be:

  1. Modules 5 and 2 will be tested thoroughly by making sure all the test cases related to them are tested. The other modules will be tested on an exploratory basis.
  2. Modules 5 and 2 are going to be tested first and then depending on the time available the others are going to be taken care of.

Once a plan is made, all the teams reach an agreement and follow it to test the product keeping into account the risk-factor.

That’s it!

A few important points to note about Risk Based Testing:

Please let us know your comments and questions on this topic below. Also, have you used this formal process in your projects? If yes, please share your experiences.