How to Set Defect Severity and Priority: Defect Triage Process

In this tutorial, you will learn what is Defect Severity and Priority, How to set it and Defect Triage process with many examples to understand the concept clearly.

We will also cover in detail on how to classify the defects under different buckets and their relevance in defect life cycle. We will also cover the crucial role of the classification with the live set of examples. 

Filing defects is a very integral part of the Software Testing Life Cycle. There are several best practices defined for effective defect reporting over the internet or in organizations.

One of the important aspects of the defect life cycle on a generic level includes defect tracking. The reason this is important is because test teams open several defects when testing a piece of software which is only multiplied if the particular system under test is complex. In such a scenario, managing these defects and analyzing these defects to drive closure can be a daunting task.

In line with defect maintenance processes, when any tester files a defect- apart from the method/description to reproduce the issue seen, he has to also furnish some categorical information which would aid in accurate classification of the defect. This, in turn, would help in efficient defect tracking/maintenance processes and would also form the basis for quicker defect turnaround time.

 

The two main parameters that form the basis for effective defect tracking and resolution are:

These are often a confused concept and are almost used interchangeably amongst not only test teams but also development teams. There’s a fine line between the two and it’s important to understand that there are indeed differences between the two.

Let’s understand briefly the theoretical definitions of the two parameters in the next section.

What You Will Learn:

What is Bug Priority and Severity?

Priority by the English definition is used in the comparison of two things or conditions, where one has to be given more importance than the other(s) and has to be tackled with/resolved first before proceeding to the next one(s). Therefore in the context of defects, the priority of a defect would indicate the urgency with which it would need to be fixed.

Severity by the English definition is used to describe the gravity of an undesirable occurrence. Hence when it comes to bugs, the severity of a bug would indicate the effect it has on the system in terms of its impact.

Bug Priority and Severity levels

Priority and Severity have some classifications amongst them that aid in determining how the defect must be handled. A lot of different organizations have different defect logging tools, so the levels might vary.

Let’s take a look at the different levels for both Priority and Severity.

What’s the difference between priority and severity?

Priority is associated with scheduling, and “severity” is associated with standards.

“Priority” means something is afforded or deserves prior attention; a precedence established by order of importance (or urgency).

“Severity” is the state or quality of being severe; severe implies adherence to rigorous standards or high principles and often suggests harshness; severe is marked by or requires strict adherence to rigorous standards or high principles, e.g. a severe code of behavior.

The words priority and severity do come up in bug tracking.

A variety of commercial, problem tracking/management software tools are available. These tools, with the detailed input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its ‘severity’, reproduce it and fix it.

The fixes are based on project ‘priorities’ and ‘severity’ of bugs.

The ‘severity’ of a problem is defined in accordance with the customer’s risk assessment and recorded in their selected tracking tool.

A buggy software can ‘severely’ affect schedules, which, in turn, can lead to a reassessment and renegotiation of ‘priorities’.

Priority:

As already discussed priority determines how quickly the defect turnaround time must be. If there are multiple defects, the priority decides which defect has to be fixed and verified immediately versus which defect can be fixed a bit later.

While opening a defect, the tester generally assigns the priority initially as he views the product from the end user perspective. In line with these, there are different levels:

 

Severity:

Severity is a parameter to denote the implication of defect on the system – how critical defect is and what is the impact of the defect on the whole system’s functionality? The severity is a parameter set by the tester while he opens a defect and is mainly in control of the tester. Again different organizations have different tools to use for defects, but on a generic level these are the following severity levels:

Examples of sorting defects with priority and severity

Setting the defect priority and severity examples

As already mentioned, since different organizations use different kinds of tools for defect tracking and its related processes- it becomes a common tracking system between various levels of management and the technical personnel.

Since defect severity is more within the purview of the functionality, the test engineer sets the severity of the defect. At times the developers part-take in influencing the defect severity, but mostly it’s dependent on the tester as he evaluates how much a particular feature can impact the overall functioning.

On the other hand, when it comes to setting defect priority, although initially, the defect originator sets the priority, it is actually defined by the Product Manager as he has an overall view of the product and how quickly a particular defect has to be addressed. A tester is not an ideal person to set the defect priority.

Shocking as this may seem, there are two distinct examples as to why:

Example #1: Consider that there is a situation where the user finds a mistake in the naming of the product itself or some problem with the UI documentation. A tester would normally open a minor/cosmetic defect and may be very simple to fix, but when it comes to the product look and feel / User experience, it could cause a serious impact.

Example #2: There could be certain conditions under which a particular defect occurs which may be an extremely rare or no possibility to hit in the customer environment. Even though functionality-wise this may seem like a high priority defect to a tester, considering its rarity of occurrence and high cost to fix – this would be classified as a low priority defect.

Hence in effect, the defect priority is generally set by the product manager in a “defect triage” meeting, which is discussed in the next segment.

Defect Triage

A commonly heard and experienced situation in test teams is limited availability of resources. Defect triage is a process which tries to do some re-balancing as a result of this phenomenon. So when there are many defects and limited Developers/testers to fix/verify them, defect triage helps to get as many defects resolved as possible by balancing the technical personnel based on defect parameters like priority and severity.

Typically, a defect triage session is attended by the Product Manager, a development lead, a test lead and sometimes business analysts. In some cases, certain other members may also be invited to give their opinions and perspectives regarding certain defects. These are collectively called a triage team.

Defect Triage Process:

Most systems use priority as the main criteria to assess the defect, however, a good triage process considers the severity as well.

Let’s take a closer look at the triage process with two examples that we’ve talked about in the previous section. In both the examples above, it would actually be the first defect that would be given a very high priority. Despite it being only a cosmetic defect, the impact of not fixing would be huge.

The second one, on the other hand, is a surely functionality defect, however, its occurrence is in only certain conditions which are seldom practised customer scenarios. Fixing it may need more time and people, which could be better utilized for other defects. Hence it would deem lower priority than that of the first and maybe deferral candidate to another release.



Thus the triage process involves triage team sitting down together, reviewing all the defects including rejected defects. They draw an initial assessment of the defects based on its content, their respective priority, and severity settings; with each person in the triage team presenting their perspective on how to prioritize the defects. The product manager then sets the priority based on all the inputs and assigns the defect to the correct release I.e. in the current release or any future release. He also redirects the defect to the correct owner/team for further action. Rejected defects also are put through a similar analysis. Based on the reason for rejection, the futuristic action of whether it needs to be deferred or cancelled is determined.

Suggestions to choose defect severity and priority correctly

As we’ve already discussed, the severity parameter is assessed by the tester whereas the priority parameter is mainly assessed by the Product Manager or basically the triage team. Even while this is the case, the severity of a defect is definitely one of the governing and influencing factors for the prioritizing the defect. Hence it’s important as a tester to select the right severity to avoid confusion with development teams.

Below are certain guidelines that every tester must try to follow:

 What is Severity?

Severity defines the extent to which a particular defect could create an impact on the application or system.

For Example: Consider the following scenarios

What would be the user experience, if any of the above scenarios could happen?

Broadly the defects can be classified as follows:

#1) Critical

For any reason, if the application crashes or it becomes unusable / not able to proceed further, the defect could be classified under critical severity.

Any catastrophic system failures could lead the user to non-usability of the applications could be classified under Critical severity

For example: In the email service provider like Yahoo or Gmail, after typing the correct Username and the password, instead of logging in, the system crashes or throws the error message, this defect is classified as critical as this defect makes the whole application unusable.

The scenario on point 1 discussed above could be classified as Critical Defect, as the online application becomes completely unusable.

#2) Major

Any Major feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected, it can be classified under Major Severity.

For example: In the email service provider like Yahoo or Gmail, when you are not allowed to add more than one recipient in the CC section, this defect is classified as the Major defect as the major functionality of the application is not working properly. What is expected the behaviour of the CC section in the mail, it should allow the User to add multiple Users. So when the major functionality of the application is not working properly or when it behaves differently than expected, it is major defect,

The scenarios on point 2 & 3 discussed above could be classified as Major Defect, as the order is expected to move smoothly to the next phase of order life cycle but in reality, it varies in behaviour

Any defect that could lead to incorrect data persistence, data issues or wrong application behaviours could be broadly classified under the Major severity

#3) Minor

Any feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected but the impact is negligible to some extent or it doesn’t have major impact on the application, can be classified under Minor Severity

For example: In the email service provider like Yahoo or Gmail, there is option called “Terms and Conditions” and in that option there will be multiple links regarding the terms and condition of the website, When one among the multiple links, is not working fine, it is called as Minor severity as it only affects minor functionality of the application and it doesn’t have big impact on the Usability of the application.

The scenario on point 5 discussed above could be classified as Minor Defect, as there is no data loss or failure in system flow order but slight inconvenience when it comes to user experience

These types of defect result in minimal loss of functionality or user experience

#4) Low

Any cosmetic defects including spelling mistakes or alignment issues or font casing can be classified under Low Severity

For example: In the email service provider like Yahoo or Gmail, You would have noticed the “License page”, if there is any spelling mistakes or misalignment in the page, this defect is classified as Low.

The scenario on point 6 discussed above could be classified as Low Defect, as the Add button is displayed in wrong Casing. This kind of defects will not have any impact on system behaviour or data presentation or data loss or data flow or even user experience but will be very cosmetic.

******************

Update: Below section is an addition to the content we already have on this topic. Below we have added more examples for quick understanding of the topic.

What is Priority?

Priority, as the name suggests, is about prioritizing a defect based on business needs and severity of the defect. Priority signifies the importance or urgency of fixing a defect.

Broadly, Priority of the defects can be classified as follows:

#1) Immediate/Critical

Any defect that needs immediate attention which impacts the testing process will be classified under the immediate category

All the Critical severity defects fall under this category (unless re-prioritized by business/stakeholders)

#2) High

This is the defect or issue which should be resolved before the release is made. These defects should be resolved once the Critical issues are solved.

All the Major severity defects fall under this category.

#3) Medium

This defect should be resolved after all the serious bugs are fixed.

Once the Critical and the High priority bugs are done, we can go for the medium priority bugs.

All the Minor severity defects fall under this category.

#4) Low

This defect can be resolved in future and does not need any immediate attention and the Low severity defects fall under this category.

To summarize, the following figure depicts the broad classification of software defects based on Severity and Priority

A practical scenario for classification

Typically the defects could be classified under one of the following categories

The following figure depicts the classification of the categories in a single snippet.

#1) High Severity and High Priority

Any Critical/major business case failure automatically gets promoted to this category.

For example:

The system crashes after you made the payment or when you are not able to add the items to the Cart, this defect is marked as High Severity and High Priority defect.

Another example would be ATM vending currency feature wherein after entering the correct username and the password, the machine does not dispense money but deducts the transferred from your account.

#2) High Priority and Low severity

Any minor severity defects that could directly impact the user experience automatically gets promoted to this category.

For example:

The logo of the company in the front-page is wrong, it is considered to be High Priority and Low severity defect.

Example 1: In the Online shopping website, when the FrontPage logo is spelt wrong, for example instead of Flipkart it is spelt as Flipkrta.

Example 2: In the bank logo, instead of ICICI, it is written as ICCCI.

In terms of functionality, it is not affecting anything so we can mark as Low Severity, but it has an impact on user experience. This kind of defects needs to be fixed on high priority even though they have very less impact on the application side.

#3) High Severity and Low Priority

Any defect that is functionally not meeting the requirements or have any functional implications on the system but sidelined to back seat by the stakeholders when it comes to business criticality automatically gets promoted to this category.

For example:

In a social networking site, if a beta version of a new feature is released with not many active users using that facility as of today. Any defect found on this feature can be classified as a Low priority as the feature takes back seat due to business classification as not important.

Though this feature is having a functional defect, as it is not impacting the end customers directly, a business stakeholder can classify the defect under low priority though it has a severe functional impact on the application.

This is a high severity fault but can be prioritized to low priority as it can be fixed with next release as a change request. Business stakeholders also prioritize this feature as a rarely used feature and do not impact any other features that have a direct impact on user experience. This kind of defects can be classified under High severity but Low Priority category.

#4) Low Severity and Low priority

Any spelling mistakes /font casing/ misalignment in the paragraph of the 3rd or 4th page of the application and not in the main or front page/ title.

For Example:

If the private policy of the website has a spelling mistake, this defect is set as Low Severity and Low priority.

Who defines Severity & Priority

QA classifies the defect under appropriate severity based on the complexity and criticality of the defects.

Any business stakeholders including the project managers, business analysts, product owner define the priority of the defects.

The below figure depicts the role who owns & classifies the criticality & severity of the defects.

Conclusion

While opening defects it’s a tester’s responsibility to assign the right severity to the defects. Incorrect severity and hence priority mapping can have very drastic implications on the overall STLC process and the product as a whole. In several job interviews – there are several questions that are asked on priority and severity to ensure that as a tester you have these concepts impeccably clear in your mind.

Also, we had seen live examples on how to classify the defect under various Severity / Priority buckets. By now, I wish you had enough clarification on defect classification both at severity/priority buckets.

About Author: This is a guest post by Sneha Nadig. She is having 7+ years of software testing experience and currently working as a test lead in a top MNC.

Hope this article is a complete guide to understanding the Defect triage process for defect priority and severity settings. Let us know your thoughts/questions in comments below.