In this tutorial, you will learn what is Defect Severity and Priority in testing, how to set defect priority and severity levels with examples to understand the concept clearly.
We will also cover in detail how to classify the defects under different buckets and their relevance in the Defect Life cycle. We will also cover the crucial role of the classification with a live set of examples.
Filing defects is a very integral part of the Software Testing Life Cycle. There are several best practices defined for effective Defect Reporting over the internet or in organizations.
What You Will Learn:
Defect Tracking Overview
One of the important aspects of the Defect Life cycle on a generic level includes defect tracking. This is important because test teams open several defects when testing a piece of software which is only multiplied if the particular system under test is complex. In such a scenario, managing these defects and analyzing these defects to drive closure can be a daunting task.
In line with defect maintenance processes, when any tester files a defect- apart from the method/description to reproduce the issue seen, he has to also furnish some categorical information that would aid inaccurate classification of the defect. This, in turn, would help in efficient defect tracking/maintenance processes and would also form the basis for quicker defect turnaround time.
The two main parameters that form the basis for effective Defect Tracking and Resolution are:
- Defect Priority in Testing
- Defect Severity in Testing
These are often a confusing concept and are almost used interchangeably amongst not only test teams but also development teams. There’s a fine line between the two and it’s important to understand that there are indeed differences between the two.
Let’s understand briefly the theoretical definitions of the two parameters in the next section.
What Is Defect Severity And Priority?
Priority by the English definition is used in the comparison of two things or conditions, where one has to be given more importance than the other(s) and has to be tackled with/resolved first before proceeding to the next one(s). Therefore in the context of defects, the priority of a defect would indicate the urgency with which it would need to be fixed.
Severity by the English definition is used to describe the gravity of an undesirable occurrence. Hence when it comes to bugs, the severity of a bug would indicate the effect it has on the system in terms of its impact.
Who Defines These?
QA classifies the defect under appropriate severity based on the complexity and criticality of the defects.
Any business stakeholders including the project managers, business analysts, product owner define the priority of the defects.
The below figure depicts the role who owns & classifies the criticality & severity of the defects.
How To Choose These Levels?
As we’ve already discussed, the severity parameter is assessed by the tester whereas the priority parameter is mainly assessed by the Product Manager or basically the triage team. Even while this is the case, the severity of a defect is definitely one of the governing and influencing factors for prioritizing the defect. Hence it’s important as a tester to select the right severity to avoid confusion with development teams.
Difference Between Severity And Priority
Priority is associated with scheduling, and “severity” is associated with standards.
“Priority” means something is afforded or deserves prior attention; precedence established by order of importance (or urgency).
“Severity” is the state or quality of being severe; severe implies adherence to rigorous standards or high principles and often suggests harshness; severe is marked by or requires strict adherence to rigorous standards or high principles, For Example, a severe code of behavior.
The words priority and severity do come up in bug tracking.
A variety of commercial, problem tracking/management software tools is available. These tools, with the detailed input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its ‘Severity’, reproduce it and fix it.
The fixes are based on project ‘Priorities’ and ‘Severity’ of bugs.
The ‘Severity’ of a problem is defined in accordance with the customer’s risk assessment and recorded in their selected tracking tool.
Buggy software can ‘severely’ affect schedules, which, in turn, can lead to a reassessment and renegotiation of ‘priorities’.
What Is Priority?
Priority, as the name suggests, is about prioritizing a defect based on business needs and severity of the defect. Priority signifies the importance or urgency of fixing a defect.
While opening a defect, the tester generally assigns the priority initially as he views the product from the end-user perspective. In line with these, there are different levels:
Broadly, Priority of the defects can be classified as follows:
Priority #1) Immediate/Critical (P1)
This has to be fixed immediately within 24 hours. This generally occurs in cases when an entire functionality is blocked and no testing can proceed as a result of this. Or in certain other cases if there are significant memory leaks, then generally the defect is classified as a priority -1 meaning the program/ feature is unusable in the current state.
Any defect that needs immediate attention which impacts the testing process will be classified under the immediate category
All the Critical severity defects fall under this category (unless re-prioritized by business/stakeholders)
Priority #2) High (P2)
Once the critical defects have been fixed, a defect having this priority is the next candidate which has to be fixed for any test activity to match the “exit” criteria. Normally when a feature is not usable as it’s supposed to be, due to a program defect, or that new code has to be written or sometimes even because some environmental problem has to be handled through the code, a defect may qualify for a priority 2.
This is the defect or issue which should be resolved before the release is made. These defects should be resolved once the Critical issues are solved.
All the Major severity defects fall into this category.
Priority #3) Medium (P3)
A defect with this priority must be in contention to be fixed as it could also deal with functionality issues which are not as per expectation. Sometimes even cosmetic errors such as expecting the right error message during the failure could qualify to be a priority 3 defect.
This defect should be resolved after all the serious bugs are fixed.
Once the Critical and the High priority bugs are done, we can go for the medium priority bugs.
All the Minor severity defects fall into this category.
Priority #4) Low (P4)
A defect with low priority indicates that there is definitely an issue, but it doesn’t have to be fixed to match the “exit” criteria. However, this must be fixed before the GA is done. Typically, some typing errors or even cosmetic errors as discussed previously could be categorized here.
Sometimes defects with priority low are also opened to suggest some enhancements in the existing design or a request to implement a small feature to enhance user experience.
This defect can be resolved in the future and does not need any immediate attention and the Low severity defects fall into this category.
As already discussed priority determines how quickly the defect turnaround time must be. If there are multiple defects, the priority decides which defect has to be fixed and verified immediately versus which defect can be fixed a bit later.
What Is Severity?
Severity defines the extent to which a particular defect could create an impact on the application or system.
Severity is a parameter to denote the implication of defect on the system – how critical defect is and what is the impact of the defect on the whole system’s functionality? The severity is a parameter set by the tester while he opens a defect and is mainly in control of the tester. Again different organizations have different tools to use for defects, but on a generic level these are the following severity levels:
For Example, Consider the following scenarios
- If the user tries to do online shopping and the application does not load or server unavailable message pops up.
- The user performs adding an item to the cart, the number of quantities added is incorrect/wrong product gets added.
- The user makes the payment and after the payment, the order stays in the cart as reserved instead confirmed.
- The system accepts the order but finally, cancels the order after half an hour due to any issues.
- The system accepts the “Add to Cart” on double click only instead of on a single click.
- The Add To Cart button is spelled as Add To Cart.
What would be the user experience, if any of the above scenarios could happen?
Broadly the defects can be classified as follows:
#1) Critical (S1)
A defect that completely hampers or blocks testing of the product/ feature is a critical defect. An example would be in the case of UI testing where after going through a wizard, the UI just hangs in one pane or doesn’t go further to trigger the function. Or in some other cases, when the feature developed itself is missing from the build.
For any reason, if the application crashes or it becomes unusable / not able to proceed further, the defect could be classified under critical severity.
Any catastrophic system failures could lead the user to non-usability of the applications could be classified under the Critical severity
For Example, In the email service provider like Yahoo or Gmail, after typing the correct username and the password, instead of logging in, the system crashes or throws the error message, this defect is classified as critical as this defect makes the whole application unusable.
The scenario on point 1 discussed above could be classified as Critical Defect, as the online application becomes completely unusable.
#2) Major (S2)
Any Major feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected, it can be classified under Major Severity.
A major defect occurs when the functionality is functioning grossly away from the expectations or not doing what it should be doing. An example could be: Say that a VLAN needs to be deployed on the switch and you are using a UI template that triggers this function. When this template to configure VLAN fails on the switch, it gets classified as a severe functionality drawback.
For Example, In the email service provider like Yahoo or Gmail, when you are not allowed to add more than one recipient in the CC section, this defect is classified as the Major defect as the major functionality of the application is not working properly.
What is expected the behavior of the CC section in the mail, it should allow the user to add multiple Users. So when the major functionality of the application is not working properly or when it behaves differently than expected, it is a major defect.
The scenarios on point 2 & 3 discussed above could be classified as Major Defect, as the order is expected to move smoothly to the next phase of the order life cycle but in reality, it varies in behavior.
Any defect that could lead to incorrect data persistence, data issues or wrong application behaviors could be broadly classified under the Major severity.
#3) Minor/Moderate (S3)
Any feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected but the impact is negligible to some extent or it doesn’t have a major impact on the application, can be classified under Minor Severity.
A moderate defect occurs when the product or application doesn’t meet certain criteria or still exhibits some unnatural behavior, however, the functionality as a whole is not impacted. For example in the VLAN template deploy above, a moderate or normal defect would occur when the template is deployed successfully on the switch, however, there is no indication being sent to the user.
For Example, In the email service provider like Yahoo or Gmail, there is option called “Terms and Conditions” and in that option, there will be multiple links regarding the terms and condition of the website, When one among the multiple links, is not working fine, it is called as Minor severity as it only affects minor functionality of the application and it doesn’t have big impact on the Usability of the application.
The scenario on point 5 discussed above could be classified as Minor Defect, as there is no data loss or failure in system flow order but a slight inconvenience when it comes to user experience.
These types of defect result in minimal loss of functionality or user experience.
#4) Low (S4)
Any cosmetic defects including spelling mistakes or alignment issues or font casing can be classified under Low Severity.
A minor low severity bug occurs when there is almost no impact on the functionality but it is still a valid defect that should be corrected. Examples of this could include spelling mistakes in error messages printed to users or defects to enhance the look and feel of a feature.
For Example, In the email service provider like Yahoo or Gmail, You would have noticed the “License page”, if there is any spelling mistakes or misalignment in the page, this defect is classified as Low.
The scenario on point 6 discussed above could be classified as Low Defect, as the Add button is displayed in wrong Casing. This kind of defect will not have any impact on system behavior or data presentation or data loss or data flow or even user experience but will be very cosmetic.
To summarize, the following figure depicts the broad Defect classification based on Severity and Priority:
Examples
As already mentioned, since different organizations use different kinds of tools for defect tracking and its related processes- it becomes a common tracking system between various levels of management and the technical personnel.
Since Defect Severity is more within the purview of the functionality, the Test Engineer sets the severity of the defect. At times the developers part-take in influencing the defect severity, but mostly it’s dependent on the tester as he evaluates how much a particular feature can impact the overall functioning.
On the other hand, when it comes to setting defect priority, although initially, the defect originator sets the priority, it is actually defined by the Product Manager as he has an overall view of the product and how quickly a particular defect has to be addressed. A tester is not an ideal person to set the defect priority.
Shocking as this may seem, there are two distinct examples as to why:
Example #1) Consider that there is a situation where the user finds a mistake in the naming of the product itself or some problem with the UI documentation. A tester would normally open a minor/cosmetic defect and may be very simple to fix, but when it comes to the product look and feel / User experience, it could cause a serious impact.
Example #2) There could be certain conditions under which a particular defect occurs which may be an extremely rare or no possibility to hit in the customer environment. Even though functionality-wise this may seem like a high priority defect to a tester, considering its rarity of occurrence and high cost to fix – this would be classified as a low priority defect.
Hence in effect, the defect priority is generally set by the product manager in a “defect triage” meeting.
Different Levels
Priority and Severity have some classifications amongst them that aid in determining how the defect must be handled. A lot of different organizations have different defect logging tools, so the levels might vary.
Let’s take a look at the different levels for both Priority and Severity.
- High Priority, High Severity
- High Priority, Low Severity
- High Severity, Low Priority
- Low Severity, Low Priority
The following figure depicts the classification of the categories in a single snippet.
#1) High Severity and High Priority
Any Critical/major business case failure automatically gets promoted to this category.
Any defects due to which the testing cannot continue at any cost or causes a severe system failure to fall into this category. For Example, clicking on a particular button doesn’t load the feature itself. Or performing a particular function brings down the server consistently and causes data loss. The red lines in the above figure indicate these kinds of defects.
For Example,
The system crashes after you made the payment or when you are not able to add the items to the Cart, this defect is marked as High Severity and High Priority defect.
Another example would be ATM vending currency feature wherein after entering the correct username and the password, the machine does not dispense money but deducts the transferred from your account.
#2) High Priority and Low Severity
Any minor severity defects that could directly impact the user experience automatically gets promoted to this category.
Defects that have to be fixed but do not affect the application come under this category.
For Example, the feature is expected to display a particular error to the user with respect to its return code. In this case, functionally the code will throw an error, but the message will need to be more relevant to the return code generated. The blue lines in the figure indicate these kinds of defects.
For Example,
The logo of the company in the front-page is wrong, it is considered to be High Priority and Low Severity defect.
Example 1) In the Online shopping website when the FrontPage logo is spelled wrong, for example instead of Flipkart it is spelled as Flipkart.
Example 2) In the bank logo, instead of ICICI, it is written as ICCCI.
In terms of functionality, it is not affecting anything so we can mark as Low Severity, but it has an impact on user experience. This kind of defect needs to be fixed on high priority even though they have very less impact on the application side.
#3) High Severity and Low Priority
Any defect that is functionally not meeting the requirements or have any functional implications on the system but sidelined to back seat by the stakeholders when it comes to business criticality automatically gets promoted to this category.
Defects that have to be fixed but not immediately. This can specifically occur during ad-hoc testing. It means that the functionality is affected to a large extent, but is observed only when certain uncommon input parameters are used.
For Example, a particular functionality can be used only on a later version of the firmware, so in order to verify this – the tester actually downgrades his system and performs the test and observes a serious functionality issue that is valid. In such a case the defects will be classified in this category denoted by pink lines, as normally end users will be expected to have a higher version of the firmware.
For Example,
In a social networking site, if a beta version of a new feature is released with not many active users using that facility as of today. Any defect found on this feature can be classified as a low priority as the feature takes back seat due to business classification as not important.
Though this feature is having a functional defect, as it is not impacting the end customers directly, a business stakeholder can classify the defect under low priority though it has a severe functional impact on the application.
This is a high severity fault but can be prioritized to low priority as it can be fixed with the next release as a change request. Business stakeholders also prioritize this feature as a rarely used feature and do not impact any other features that have a direct impact on user experience. This kind of defect can be classified under the High Severity but Low Priority category.
#4) Low Severity and Low Priority
Any spelling mistakes /font casing/ misalignment in the paragraph of the 3rd or 4th page of the application and not in the main or front page/ title.
These defects are classified in the green lines as shown in the figure and occur when there is no functionality impact, but still not meeting the standards to a small degree. Generally cosmetic errors or say dimensions of a cell in a table on UI are classified here.
For Example,
If the privacy policy of the website has a spelling mistake, this defect is set as Low Severity and Low Priority.
Guidelines
Below are certain guidelines that every tester must try to follow:
- Firstly, understand the concepts of priority and severity well. Avoid confusing one with the other and using them interchangeably. In line with this, follow the severity guidelines published by your organization/team so that everyone is on the same page.
- Always choose the severity level based on the issue type as this will affect its priority. Some examples are:
- For an issue which is critical, such as the entire system goes down and nothing can be done – this severity should be not be used to address program defects.
- For an issue which is major, such as in cases where the function is not working as expected – this severity could be used to address new functions or improvement in the current working.
Remember, that choosing the right severity level will, in turn, give the defect, it’s the due priority.
- As a tester – understand how a particular functionality, rather drilling down further – understand how a particular scenario or test case would affect the end-user. This involves a lot of collaboration and interaction with the development team, Business Analysts, architects, Test lead, Development lead. In your discussions, you also need to factor in how much time it would take to fix the defect based on its complexity and time to verify this defect.
- Finally, it’s always the product owner who possesses the veto power of the release the defect should be fixed. However since the defect triage sessions contain varied members to present their perspective on the defect on a case basis, at such a time if the developers and testers are in sync, it surely helps in influencing the decision.
Conclusion
While opening defects it’s a tester’s responsibility to assign the right severity to the defects. Incorrect severity and hence priority mapping can have very drastic implications on the overall STLC process and the product as a whole. In several job interviews – there are several questions that are asked about priority and severity to ensure that as a tester you have these concepts impeccably clear in your mind.
Also, we had seen live examples of how to classify the defect under various Severity / Priority buckets. By now, I wish you had enough clarification on defect classification both at severity/priority buckets.
Hope this article is a complete guide to understanding the defect priority and severity levels. Let us know your thoughts/questions in the comments below.
thanks for sharing. this is really a good guide.
one question in our project we QAs set both the severity and prio and that is not revised by TM. Do we need to change this process?
Thanks a lot for sharing this article……..One question is there……In JIRA is there any option to set the Severity for the issues which we will create?
Hi, Nikita
not at this moment (may ’20). I needed to create a Custom Field for my company’s Jira Project.
Hi could you please take a popular interview question, that will be a great example and help us in understand the concept.
Give example of:
LS LP
LS HP
HS LP
HS HP scenarios for a “White Board Marker” ?
HP HP scenario for white board marker
– unable to write through marker in which so main functionality not working.
LP – LS
Brand name name logo of marker wrongly spleeled.
LS – HP
Cap of marker not perfectly fit (intergartion testing) and colour of marker wrongly mention in sticker.
HS – LP
After writing through marker unable to erase it but in sticker it is mentioned it eraseble.
can you explain using loging page
Really very nice article. I am impressed.
It’s easy to understand the differences.
Thanks a lot.
Thank you all for your feedback.
@Megha : Yes there are several organizations having different tools for defect handling. Setting priority and severity maybe a mandatory parameter to even open a defect, in which case the tester sets them both. However, rest assured that your program management will be monitoring all your defects for Quality purposes and will definitely refine the priority as it makes sense. This is especially more apparent when there are very tight schedules, limited development and test resources and a large number of defects.
@Nandini : It would help if you put in the scenario. I’m sorry I don’t seem to catch anything from just “white board marker”
@Nikita : Sorry, I have not used JIRA. So I would defer this question to any of the readers that may have used the tool.
Thanks for sharing. This is a straight forward article that brings out the difference between defect priority and severity and easy to understand.
Thanks for guidance…
I would like to pin-point the line on Priority 4, you have mentioned ‘GA’. Kindly elaborate what do you mean by it.
I think it is a typo.
Rest was good explanation.
Nice article,regarding Anuroop question,its a typo..it was ‘GI’…Thanks for sharing.
@Anuroop:GA denotes General Availability where by the software is available to the end users via the web. It’s not a typo. However I have to agree that I should have elaborated it. Thank you for your keen observations. It’s a good learning for me as well.
amazing article thank you for clarifying the concepts.
Very good article but another way of looking at it would be from the test phase for example:
Severity:
• Severity is set based on the technical aspect of the failure during all test phases.
Priority:
• During SIT – set to indicate the fix order of the defects, when there are multiple defects for any given severity level.
• During UAT – set based on the business requirement.
• Combined SIT/UAT – set based on QA and Business agreement.
Development Fixed and Delivered
• During SIT the development team will fix defects based on severity and then priority.
• During UAT or combined SIT & UAT the development team will fix defects based on Priority.
I have been in the testing field for over 20 years, using many different testing tools and in many different organizations both public and private from which I have developed a way to define these fields based on the test phase that has been very affective and that both business and development teams can agree with.
I have the severity number for the test requirement how to identify what is the defect type i.e., whether it is Critical or major or minor.
Good artcile
Hi All, just to add few exta stuffs regarding this content.
There is another defect tracking tool called, ALM-Application Lifecycle Management where the Priority & Severity as categorised as below:
Priority Severity
1-Urgent 1-Extreme
2-Very High 2-Major
3-High 3-Significant
4-Medium 4-Minor
5-Low 5-Cosmetic
Hi,
I have one query regard to bug severity.
Suppose, I found an issue while testing, I will raise a issue in the tool by mentioning the severity as Major (ex). In case with the same sw/build/package/product, If I couldn’t able to reproduce the issue again, do we need to change the severity from Major to Minor?
Please provide your views on this.
Thanks.
Mahantesh
Priority should be set by the business, not the tester. A tester is never in a position to tell the customer which bugs to fix first, unless he is also the customer.
can developer change the priority
@Raviteja, Yes Development lead can set the priority, if they closely work with Business team and knows the customer requirement very well.
@Mahantesh, Yes if the issue is not reproducible than you need to change the severity, else you could mark a level saying that, issue is in-consistent. Again the process is purely varies on project to project.
@Yogaraj S, Here you go. it might helps you.
Severity:
It is the extent to which the defect can affect the software. In other words it defines the impact that a given defect has on the system. For example: If an application or web page crashes when a remote link is clicked, in this case clicking the remote link by an user is rare but the impact of application crashing is severe. So the severity is high but priority is low.
Severity can be of following types:
Critical: The defect that results in the termination of the complete system or one or more component of the system and causes extensive corruption of the data. The failed function is unusable and there is no acceptable alternative method to achieve the required results then the severity will be stated as critical.
Major: The defect that results in the termination of the complete system or one or more component of the system and causes extensive corruption of the data. The failed function is unusable but there exists an acceptable alternative method to achieve the required results then the severity will be stated as major.
Hi Sneha.. Thanks for sharing such a valuable information with us.. I have a query regarding the same topic..
My query is I need to find the Weighted Defect Density of few bugs. I searched on different sites, though the formula all over the sites is same yet the weightage given to the different severities differ i.e in my case we have classified severity into Complex, Medium & Low. In some websites the three are weighted as 3,2 & 1 and in some they are weighted as 5, 3 & 1.
Please suggest the correct method and the correct value of the weights to be used.
Hii, Nikita
Yes we can set the severity of the bug in the JIRA.
Hi nikita mam,
Your article really help me to resolved all doubt about priority &severity.
Thanks a lot
Hi mam,
Your article really help me to resolved all doubt about priority &severity.
Thanks a lot
awesome thanks for sharing……..
Do these categories for Severity and Priority come from any known standards like IEEE or MIL-STD? Or are they just defined in commercial best practices and tools?
My answer is too sneha mam,what is workaround in an application?
It’s that mean A temporary fix available or provided of whatever the problem is occuring inside the application.
Hi, Sneha.
Thanks a lot for your efforts in writing that comprehensive and concrete article – I’ve never made any difference between Severity and Priority, though it’d be awesome improvement in the project organization.
A few questions if you don’t mind checking and replying:
1. In section “Suggestions to choose defect severity and priority correctly” is everythin okay with the phrase
“this severity should be not be used to address program defects.” ?
Sounds like a misprint to me.
2. Under “What is Severity?” paragraph have you assigned the following issue:
“The system accepts the order but finally, cancels the order after half an hour due to any issues.”
to any Severity ?
TIA,
Pavel
Hi,
In JIRA we are setting Priority and Fix Priority. Are they different. Fix priority means Severity?
I’m going to comment from the perspective of user experience, customer experience, and business reputation. Alignment issues, misspellings, and poor grammar, from the perspective of engineers, are considered “merely” cosmetic and of low priority. From the point of view of a customer using the product, a page with content out of alignment and text with numerous typos and poor grammar raises doubt about the credibility of the company. (There are studies to back this up.) It sends a poor message about the company: They didn’t take the time to proofread or make sure the details of the UI are right, what ELSE haven’t they attended to?
This is the issue to me in the whole Agile, engineering focused world: you can write all the stories you like using the user story format, but when it comes down to it, the priority is building working software, not building the best product. It’s saying that the things that matter to designers and marketing are of low priority. Low priority=we’ll never get around to fixing it. But, if only designers would learn to code…
I was asked below question in interiview.Could anyone please help me with this:
Q- Mention High Severity and low priority scenario for google search results?
what is trache defect? I even dont know the spelling exactly.
it was asked in interview
If there are 10 P1 S1 issues, how do you determine which one to pick first? Since all of them are of high priority and high severity, which one would be ‘First Among Equals’?
Hi,
Thanks!. Article is very informative. But I have one question in my mind does anyone not find any defect of “Medium” priority and “High” severity? or vice versa….
Hi, It is a wonderful article on testing overview.
can you please provide us with some more examples to understand the topic of assigning the severity and priority to defects .
this one very useful and easy understanding