Failure Mode and Effects Analysis (FMEA) – How to Analyze Risks for Better Software Quality & Satisfied Customers!

Failure Mode and Effects Analysis (FMEA) is a Risk Management technique.

If implemented properly, this can be a great addition to the best Quality Assurance processes to be followed. In this article, our goal is to introduce you to this Risk Analysis technique which in the end, is very useful for improving the software quality.

Failure Mode And Effects Analysis

FMEA is mostly used by upper management or stakeholders. In practice, the testers get little insight into this technique. But now the trend is changing and I feel if the testers understand this concept properly, they can drive their thought process of writing test cases

• Understand the stakeholder’s goals of testing the application.
• Understand the business.
• Derive the high-level test scenarios based on business and management interest.
• Derive effective test cases that provide better coverage to the risk-prone areas.
• Prioritize the test cases.
• Decide what to test and what to defer at any phase.
  

Background

RISK ANALYSIS is a crucial aspect of Test Management. The question then arises – What is Risk Analysis? And why is it important? To understand this, it is vital to understand – what is RISK?

See Also => Types of Risks in Software Projects.

RISK as its literal meaning is a possibility of a negative or undesirable outcome or event. Risks if not handled or managed properly may lead to poor quality, unsatisfied customers and sometimes loss of business.

Risk has 2 attributes:

• Probability
• Impact

Probability means the chances of a particular risk to occur and impact means the extent of the effect of the risk.

What Is Risk Analysis?

Risk Analysis is a mechanism by which the identified potential risks are analyzed & studied thoroughly in order to find the probability and impact. It is advisable to measure the two attributes and based on the result we identify:

• What to test first?
• What to test more?
• What not to test(This time)?

There are many methods of doing the Risk Analysis and they are broadly classified into two types:

• Informal Techniques: These are based on experience, judgment, and intuition.
• Formal Techniques: Identifying and weighing risk attributes.

Failure Mode And Effects Analysis (FMEA): This is a formal method of doing a Risk Analysis. In the following sections, I will be discussing more on FMEA and try to elaborate it with the example.

FMEA is a formal technique of doing Risk Analysis. It is a systematic and quantitative tool in the form of a Spread Sheet that assists the members to analyze what might get wrong. To do the FMEA we require the right people on the table. It requires a representative from all walks of the industry including customers.

Description

FMEA starts and continues with Brainstorming sessions. Participants need to identify all the components, modules, dependencies, limitations that could fail in a production environment and eventually lead to poor quality, reliability and may result in loss of business.

During FMEA we not only identify the extent of the loss but also try to identify the cause of those failures. To measure FMEA, we require 3 attributes:

• Severity of the Failure (S)
• Priority of the Failure (P)
• Likelihood of the Failure (L)

We put each of these attributes in a scale shown below:

Severity Scale:

Priority scale:

Likelihood scale:

All these three attributes (Severity, Priority, and Likelihood) are individually measured in scale and then multiplied to get a Risk Priority Number (RPN).

i.e. Risk Priority Number (RPN) = S*P*L

Based on this RPN value, we determine the extent of testing. Lesser is the RPN, higher is the Risk.

Let’s try to understand it with an example:

Failure Mode Effect Analysis Example

(This is a hypothetical example only for an understanding purpose. Actual implementation and features may vary)

Let’s consider a simple example of a banking application that has 4 features.

• Feature 1: Withdraw
• Feature 2: Deposit
• Feature 3: Home Loan
• Feature 4: Fixed Deposits.

A Risk Analysis team is formed which consists of the Bank manager, UAT Test Manager ( representing end-user), Technical Architect, Test Architect, Network Administrator, DBA, and Project Manager.

After a series of brainstorming sessions, the team came up with the following Risks:

• Complex Business logic in the case of calculating the interest rate of the home loan.
• The system fails at 200 concurrent users.
• The system fails to handle documents that are more than 6 MB.

Now let’s try to calculate the Severity, Priority, and Likelihood of these identified risks.

Severity:

Priority:

Likelihood:

Now let’s put all these attributes together:

Now let’s calculate the Risk Priority Number (RPN = Severity * Priority * Likelihood)

Now the key is: Lower is the RPN – Higher is the risk.

So here for this particular example, Feature 1 (Complex business logic in case of calculating the interest rate of the home loan) has the highest risk, and feature 2 (System fails at 200 concurrent users) has the lowest risk.

How to use this to derive test cases?

Since Feature 1 is the riskiest feature, the test cases should be rigorous and more in-depth. Write the test cases to cover complete functionality and affecting modules by the feature. Use all sorts of test case writing techniques (Equivalence Partitioning and BVA, Cause and effect graph, State Transition diagram) to derive the test cases.

The test cases should not only be functional but also non-functional (Load test, Stress, Volume test, etc.). Basically, we need to do exhaustive testing of this particular feature, so base your test cases accordingly. Also, consider all the dependent modules on this important feature.

Feature 2 is the LEAST RISKY feature, so base your test cases on the major functionality. Just high-level test cases to validate that the feature works as expected should be sufficient.

Feature 3 is a MODERATE RISK feature, so base your test cases to cover all the major and dependent functionality. Write some BVA test cases to validate a few negative scenarios as well. The extent of the test cases should be between High risk and Low-risk factors. If required, include a few non-functional test cases as well.

FMEA And Degree Of Testing

Based on the RPN value, we determine the extent or degree of testing to be done.

Normally if:

• RPN is between 1-10, we do Extensive Testing (Covering in and out of the feature/module)
• RPN is between 11-30, we do Balanced Testing ( Covering all the major functionality of the feature/module)
• RPN is between 31-70, we do Opportunity testing (Covering the basic functionality of the feature/module)
• RPN is more than 70 – No testing or when time permits, only anomaly reporting.

These ranges or numbers are not restricted to the ones I mentioned above. They may vary as per the nature of the project.

Resources: Download FMEA Software and FMEA Template.

Conclusion

Risk Analysis using FMEA requires time and experience. Desired results can be achieved only by equal participation from all the responsible team members. Though this technique is formal, it requires a series of brainstorming sessions and it is equally important to document all the identified risks.

Since most of the applications are exclusive, the scale to measure the parameters of FMEA (i.e. Priority, Severity, and Likelihood) also depends on the application. If done appropriately, there are many advantages to the FMEA technique. It can be used for identifying potential risks and based on this team can plan an effective mitigation strategy.

About the Author: This is a guest article by Shilpa Chatterjee Roy. She is working in the software testing field for the past 8.5 years in various domains.

If you have used this technique please feel free to comment on your experience below.