Dolly: The more I read on the net about Bug Severity and Bug Priority the more I am getting confused. Everyone out there seems to have his own way of describing it and it leaves me even more confused. Can you help?Victor: Why do we have two fields in a defect tracker to fill in - Severity and Priority? Are not they all the same? I have noticed that when a defect is of high severity usually it is also assigned with a top priority and vice versa. If they mean the same, why do we have 2 different names and 2 different fields in the defect tracker? Why do we need these two different parameters (severity and priority) for a defect? Can't we do only with one?
Krishna Prasad: I want to know about Priority and Severity of software bugs. Please clarify my doubt with one example of each for the following:
1. Low Severity and Low Priority
2. Low Severity and High Priority
3. High Severity and High Priority
4. High Severity and Low Priority
Once again I am back with few FAQs from my mailbox. And this time they are regarding the age-old question of Bug Severity VS. Bug Priority. I can imagine how confusing it can become for someone new to the S/W testing field. But before I can try to offer any kind of solution to dismiss this confusion, kindly allow me to present you with a small story!
One afternoon at a theater:
One fine Saturday morning when Ashley (the female protagonist of my story) is still enjoying the warmth of her bed, much to her annoyance, the cell phone rings breaking the silence of the room; as if yelling to make her realize that its 8AM already and the world out side her hostel room is up and on its feet. Lazily she reaches out for her cell phone and as she looks at the screen a smile flashes across her face, even without her knowledge. It’s her BF on the phone. He asks if she can bunk her weekend MBA class and watch the noon show of the latest blockbuster in the theater near to her hostel! Ashley thinks for a moment and nods in affirmative with the typical unforgettable blushing smile flashing across her face. :) Everything seems to happen as per the plan [telling lies to her roomies, bunking the important class, reaching the multiplex in time; something that does not happen quite often :)...] until, to her horror and surprise she finds out in the theater that the person sitting behind her is none other than her own cousin who also stays in the city! Oops…
Moral of the story:
How would you describe the situation (in terms of severity)? Critical, Major, Normal, Minor... Well, assuming that Ashley’s cousin is unaware of the love story I would imagine that she would probably consider this as a Critical situation. Won’t she? Now considering that we have a critical situation (bug/defect) at hand... How fast she would want to fix this up (either by taking her cousin on a Lunch and begging her not to tell about it to anybody or by simply trying to get even with her and threaten her saying that if she opens her mouth about it she would doom her by telling about the cousin's own secret love story to others) describes the priority! The more desperate Ashley is to fix this situation up, the higher the priority! Isn't it? :)
Bug Severity:
WordWeb dictionary defines the word “Severity” as - Used of the degree of something undesirable, something hard to endure, extreme plainness. Severity of a defect/bug tells us how undesirable the defect is. For example, a bug that causes the program to crash and terminate would be considered as high severity, while a minor spelling error might be of low severity. In our organization we use a 7-point scale to rate the severity of a bug:
Blocker: This type of bug blocks development and/or testing work. Blocks users from completing the task the function was created to facilitate.
One afternoon at a theater:
One fine Saturday morning when Ashley (the female protagonist of my story) is still enjoying the warmth of her bed, much to her annoyance, the cell phone rings breaking the silence of the room; as if yelling to make her realize that its 8AM already and the world out side her hostel room is up and on its feet. Lazily she reaches out for her cell phone and as she looks at the screen a smile flashes across her face, even without her knowledge. It’s her BF on the phone. He asks if she can bunk her weekend MBA class and watch the noon show of the latest blockbuster in the theater near to her hostel! Ashley thinks for a moment and nods in affirmative with the typical unforgettable blushing smile flashing across her face. :) Everything seems to happen as per the plan [telling lies to her roomies, bunking the important class, reaching the multiplex in time; something that does not happen quite often :)...] until, to her horror and surprise she finds out in the theater that the person sitting behind her is none other than her own cousin who also stays in the city! Oops…
Moral of the story:
How would you describe the situation (in terms of severity)? Critical, Major, Normal, Minor... Well, assuming that Ashley’s cousin is unaware of the love story I would imagine that she would probably consider this as a Critical situation. Won’t she? Now considering that we have a critical situation (bug/defect) at hand... How fast she would want to fix this up (either by taking her cousin on a Lunch and begging her not to tell about it to anybody or by simply trying to get even with her and threaten her saying that if she opens her mouth about it she would doom her by telling about the cousin's own secret love story to others) describes the priority! The more desperate Ashley is to fix this situation up, the higher the priority! Isn't it? :)
Bug Severity:
WordWeb dictionary defines the word “Severity” as - Used of the degree of something undesirable, something hard to endure, extreme plainness. Severity of a defect/bug tells us how undesirable the defect is. For example, a bug that causes the program to crash and terminate would be considered as high severity, while a minor spelling error might be of low severity. In our organization we use a 7-point scale to rate the severity of a bug:
Blocker: This type of bug blocks development and/or testing work. Blocks users from completing the task the function was created to facilitate.
Critical: The software crashes, hangs, or causes you to lose data. E.g. Crashes, loss of data, severe memory leak.
Major: Major loss (or lack) of function. Users cannot do what they NEED to do, with no workarounds.
Minor: Minor loss (or lack) of function. Users cannot do what they WANT to do or a major problem where an easy workaround exists.
Trivial: Cosmetic problem like misspelled words or misaligned text. No loss of function.
Enhancement: Request for new feature or enhancement.
Unprioritized: The reporter has no opinion on the criticality of the issue.
Unprioritized: The reporter has no opinion on the criticality of the issue.
Though this can vary from organization to organization, the overall practice may remain similar. You are free to adopt any terminology that suits you as long as they describe the impact of the bugs accurately and unambiguously.
Bug Priority:
WordWeb dictionary defines the word “Priority” as - Status established in order of importance or urgency. Priority of a defect/bug tells us how soon it is required to fix the problem. Priority reflects a business decision as to how soon that bug should be fixed. Priority of the bug determines what gets fixed next and what does not. The priority of a bug can be decided from either the Project management point of view or from the user’s point of view. A bug that is of high priority from the Project management point of view may or may not be of high priority if assessed from the user’s point of view. Check with your users; their prioritizations may surprise you! Priority of a bug can be classified as:
P1 – FIX the bug ASAP before the release, preferably in the current development iteration itself.
P2 – The Bug can be fixed in the successive iteration. But the fix must be done before the next major release of the Product.
P3 – The bug can be left to be fixed in the subsequent releases.
P4 – There is no hurry. This bug can be fixed as and when time permits.
Bug Severity Vs. Bug Priority:
The severity of a bug does not necessarily translate into the urgency to fix it. A severe bug that crashes the application only once on a Feb 29th (leap year) for 0.0001% of the users is lower priority than a mishandled error condition resulting in the need to re-enter a portion of the input for every user every time. Many bugs cause crashes, but aren't fixed because the crash is very infrequent or on a version/platform/feature low on the vendor's support list. Corner-case crashes, crashes that are dependent on a rare combination of sequence of events (trigger to set off the crash) fall under such classification of low priority bugs. Once I was testing the Parser of my application. And the Parser used to crash when the parameters were: 'MaxNulls' = '999999999' and 'MaxPrefixes' = '999999999'. This was a catastrophic crash, which was causing the Program to terminate. However, it took almost 2 years to get this crash fixed as it was rated with a low Priority (despite of its high severity) for obvious reason! :)
ATM Machine Example!
Let’s take the example of an ATM machine to further understand the different possible combinations of bug severity and priority:
Low Severity and High Priority Bug: Suppose you are testing an ATM machine and you notice that the welcome screen displays a misspelled Bank name! (e.g. in place of “Welcome to ICICI Bank” it rather displays “Welcome to UCICI Bank”. This is quite possible, considering the fact that the letters “U” and “I” share close neighborhood in a typical QWERTY type keyboard). This might be a cosmetic error and would not hamper the functioning of the ATM machine in any possible way; still this could prove to be a real pain in the eyes for a customer. Considering that the spelling error is in a frequently used part of the program, it might give an overall bad impression that could hurt the Bank’s reputation. These kinds of bugs are real annoyance for the customers and no Bank would want to loose revenue for a silly error like this one. So though this bug is of low severity, its priority would be high for obvious reasons. It is quite interesting that often, low severity cosmetic errors like these get a high priority.
High Severity and Low Priority Bug: Think of a bug that cause the ATM machine to black out if an user tries to use an expired ATM card and try it for 13 consecutive times (even after the machine keeps rejecting it)! Sounds like a critical crash? Well, it might be a critical bug as far as its severity is concerned, but don’t be too surprised if you see a low Priority being assigned to it. Reason? I think its obvious!
High Severity and High Priority Bug: These kinds of bugs are easier to imagine. Any kind of bug that results in some major catastrophic failure (crashes, system hang, memory corruptions, data loss, functionalities that does not work and so on) and that occurs in areas of the application that are frequently used can be classified under such types of high severity and high priority bugs. Think of scenarios where the ATM machine - does not detect a valid card even after entering correct PIN number, hangs if tried with an invalid card and/or PIN number, blocks the card after just 1 unsuccessful entry of PIN (instead of 3 wrong entries), does not dispense money even if there is sufficient cash in the a/c and in the ATM machine, dispenses incorrect amount to the user, does not forget the previous session even after the transaction is complete (in case of ATM machines where you have to swipe your card instead of inserting it), does not return the card even after the transaction is complete and so on. As you can imagine all of these cases of bugs can fall under high severity and high priority type.
Low Severity and Low Priority Bug: These are often low impact low on urgency-meter bugs that are quite harmless and can be fixed without a hurry. These kind of bugs do not harm the application in a disastrous way and the chance of an end user/customer being annoyed by it is also very less. Think of a scenario where the ATM machine does not append a title (Mr/Miss/Mrs) to the name of the customer in the welcome screen. This would not affect the business in a drastic way. And there might not be many customers who would mind it if their name is displayed without a title.
Hope I was able to make the distinction (Bug Severity Vs. Bug Priority) clear through the examples. Now here is a small testing exercise for you. Why don’t you try and present few more examples (at least one from each combination viz. high severity and low priority bugs, low severity and high priority bugs and so on) taking the ATM machine as example?
Happy Testing…
Why is the assignment of priority and severity assumed to be the task of the tester?
ReplyDeleteI may have strong opinions about what constitutes a high priority or high severity problem with a product, but last time I checked, I wasn't running the project.
There is one kind of bug that is particularly important to me: one that blocks or slows down testing, because that problem gives other problems places and time to hide. I believe that the project manager should assign the very highest priority to bugs that will make it more difficult to find other bugs.
But I'm not running the project; the project manager is. Why are testers so keen on making decisions about the project? Why do testers accept the responsibility for making project decisions when those are business, not technical decisions?
---Michael B.
@ Michael B.,
ReplyDeleteIt is always an honor to have comments of Gurus like you on my blog posts. They make me think deeper on the subject at hand and I am sure they help my readers as much they help me. Thanks for making me think better.
Coming back to the task of the tester, I do honestly believe that the assignment of bug severity and priority should NOT be the task of a tester. Our job is to find bugs, not to rate them as if we were movie reviewers. But unfortunately, I find that often testers are expected to do this task. Who is to blame? The testers themselves who are keen to assume responsibilities, which do not belong to them? Or the stakeholders of the Project who somehow at some point of time in the past (smartly/accidentally) transferred their own responsibilities over to the testers? Honestly I don't know whom to blame here. May be, its NOT the time to start a blame game, rather the time to start educating, the testers, the Project managers and anybody who feels that assigning priorities and severity of a bug *must* be a tester's task!
I often wonder why testers feel proud to assume responsibilities, which do NOT belong to them and over which they have no control at all! E.g. assuming that "Testers ensure a Quality Product" is one such misconception. Isn't it? Your thoughts please.
Once again thanks a lot Michael for dropping by on my blog and taking your time to leave your valuable comment.
-Debasis
When a tester writes a bug report and fills in the severity and/or priority, it's just a recommendation. The project manager's job is to look over new bugs and decide whether to accept the recommendations or to change them.
ReplyDeleteFrom my experience, testers have a good idea and are correct in severity nearly all the time and in priority much of the time as well.
It is important to note that the priority of a single bug can change throughout the lifecycle of a project. Deadlines are a major influence since we sometimes reduce a showstopper (or several) to lower priority in order to make the release on time. Alternatively, if all the highest priority bugs are closed, then the next level of bugs become the new highest priority issues. Another situation I've seen is when an important customer or manager stumbles across a bug that QA thought was lower priority but the project manager feels obligated to raise the priority of it.
-Scott
I often wonder why testers feel proud to assume responsibilities, which do NOT belong to them and over which they have no control at all! E.g. assuming that "Testers ensure a Quality Product" is one such misconception. Isn't it? Your thoughts please.
ReplyDeleteFear might be a big motivator.
Let's start with the project manager. (For the record, I've been one.) The project manager could be afraid of problems in the product; or of accepting responsibility for those problems and their consequences; or of making decisions that might prevent the problems in the first place. One way of coping with these fears (not a good way, but an effective one in many cases) is to pass responsibility for all these things to someone else.
Let's move on to the developers. (For the record, I've been one.) The developers might be afraid of injecting coding errors into the program; or of misunderstanding the requirements that have been handed to them; or of telling their managers that they need more time to test the code before passing it on to the testers. One way of coping with these fears (not a good way, but an effective one in many cases) is to pass responsibility for finding the problems on to someone else.
Now let's look at the testers. (I am one.) When a project manager and the developers tell the tester, "You're responsible for the quality of the product," the tester might be afraid of politely refusing this responsibility because she's afraid of saying No; or because she doesn't see any other role that a tester could have; or because she's afraid of facing the fact that she doesn't run the project and therefore doesn't have the responsibility she thought she had. One way of coping with these fears (not a good way, but an effective one in many cases) is to accept responsibility for everything without negotiation.
These are all fears that can be addressed by some means other that Making The Tester Responsible For Everything, but we need a different mindset. Consider this: Testers are not the brains of the project; we're the antennae. We're the sensors of the project, not the processors. Unless we have control over the schedule, budget, staffing, contractual obligations, marketing, funding of the project, and sufficient information about each of them, we don't (and shouldn't) have authority over how other people do their work. (And if we do have responsibility for these things, we should properly be called project managers, not testers.)
Priority and severity are charters for development work, and as such they're management decisions, not testing decisions. Such decisions can be influenced by information that testing provides, but that's surely not the only important information in the decision. We need to remind managers (just after reminding ourselves) that we are their reporters, not their replacements. Let the managers manage the project.
@ Scott,
ReplyDeleteThanks for dropping by and for taking your time to leave your comment.
I think as long as we (testers) are assigning the severity and priority of a bug as a recommendation, it is fine. But the problem seems to emerge when testers start thinking that they are the authorized persons to finalize the status of the severity and priority of bugs that they report! As we know, our role is to provide project-critical information to the major stakeholders of the project (who are the decision-makers unlike us; the testers). Our job is to provide recommendations, not judgments. Our job is to suggest, not to try and influence the decision making!
Thanks once again for sharing your experience.
-Debasis
@ Michael B.,
ReplyDeleteIt was simply brilliant how you connected "fear" as the possible motivator behind "testers assuming responsibility of tasks that do NOT belong to them"!
I wonder if another possible motivator could be "pride"! I have seen testers taking pride in saying things like:
1. Look we are the "Quality Police" of the project!
2. We (should) have the power to block the release of the project!
3. We ensure a Quality Product!
4. We are the guardian of the Product development!
5. We must be involved in ALL the team meetings (even if they are developer's technical meetings)!
6 And so on...
While saying things like these, they (testers) often fail to realize that they are inviting trouble by stepping into traps! I must confess, once I was one such tester. And in my case, the biggest motivator behind assuming such responsibilities was "pride"! Pride is a bad thing. It takes away the ability to judge the difference between right and wrong. It can make you blind to spot traps! Though I have learnt my lesson now, how to keep aside pride and try not to take responsibility of roles that do NOT belong to me, yet I wonder if "pride" (just like "fear") could be another motivator for testers who commit such mistakes.
-Debasis
Hi Debasis,
ReplyDeleteI bumped into you blog when I was searching for some testing interview questions. All you posts are Fab (!) and I have never seen a real world 'tester' so passionate about testing. I am also a tester and you have made me fall in love with testing all over again :-)Thanks.
I think you are a tester not just by profession but also by virtue.
Keep up the good work!!
@ Mahalakshmi,
ReplyDeleteThanks for dropping by my blog. [or should I thank Google for directing you here? :)] At any rate thanks for those words of appreciation.
If I could make you fall in love with Testing all over again, then that is my biggest reward for writing this blog. Now the purpose of waking up late nights to write these articles have been fulfilled to some extent. Thanks. Do keep coming back in future to read more such posts that might help you love Testing even more. And I can assure you that you won't have to be disappointed!
Happy Testing...
-Debasis
Hi Debasis....
ReplyDeleteGood work yaar... really nice blog. Today i totally dedicated to ur blog only. Intresting one.
keep it up and continue postings yaar...
we all are always infront of u to appreciate.
I find this topic very interesting. From my experiences, I think we should only the severity status . I do believe tester have rights to block the release, but that not going to happen frequently.
ReplyDeletePriority status must be given after having discussion with team leader( person who decide project plan)
Anonymous, why do you believe that testers have rights to block the release? Who owns the project--the tester, or the project owner? Do you believe that the project owner should ignore the business factors and instead follow the tester's mandate instead?
ReplyDelete---Michael B.
cool
ReplyDeleteIts really excellent article help me a lot.
ReplyDeletemohit
Hi...Debasis..i just love to read your articles..its very helpful..i have a doubt pls clarify my doubt..
ReplyDeleteIn organizations who will assign severity and who will assign priority..please describe about this..
and i am very confused how to give high severity low priority
Just an Awesome Article!
ReplyDeletewhile reading it i thought that i am attending lecture in my class room which i miss a lot now after entering into the professional world.
Three Cheers for this Blog Writer :)!
hi debasis,
ReplyDeleteas we know priority of the defect can be change anytime in the project. Is the same is applicable to Severity also?