Testing by itself does not improve software quality. Test results are an indicator of quality, but in and of themselves, they don't improve it. Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. What you eat before you step onto the scale determines how much you will weigh, and the software development techniques you use determine how many errors testing will find. If you want to lose weight, don't buy a new scale; change your diet. If you want to improve your software, don't test more; develop better. - (Steve McConnell: “Code Complete”)
A tester reports a software bug/defect in the application he is testing. He feels that this is a genuine defect that needs attention. But to his shock and astonishment he finds out that his bug is rejected by the developer team with an excuse of “the-application-works-as-designed”! This can happen with any tester at some point in his testing career. And this can be quite frustrating too; especially if the tester feels that the software defect is a serious one and has got potential to cause severe damage to the client/end user if the software is shipped with the defect.
Having said that, not every defect that is rejected is worth fighting for. So as the first step of self-assessment, a tester might go back to the defect report that he had submitted to the programmers and verify if the defect report was well defined! Few things worth verifying in the submitted defect report are:
a) If the defect report had a well-written summary line and the steps mentioned were readable (clear and precise). Use of words might play a vital role in deciding the fate of a defect report. A single ambiguous word might suppress the seriousness of the defect and your defect report could look like a bunch of garbage, wasting the bandwidth of the defect tracking system!
b) If the defect report contained any unnecessary step(s), adding to the confusion.
c) If the report clearly mentioned what actually happened and what you expected to happen.
d) If you had mentioned the consequences of the defect, in case it is allowed to slip through the Release Phase.
e) If your tone of voice sounded sarcastic, threatening or careless in the defect report. Was it in a way, capable of making the programmer confused, unreceptive, or angry?
A well-written defect report can differentiate a best-selling bug from a flop show! If you had missed to report the defect properly, you should not blame the programmer for turning down your defect as “rejected”! May be you should spend some time on your bug/defect reporting skills and try once again. But suppose, you had reported the defect quite neatly and still it was rejected, decide if you would choose to go with the decision or rather appeal against it as you still strongly feel that this is a serious defect that needs immediate attention. In case, you are planning to appeal against the rejection these are few things that you might consider doing in order to increase your chance of success:
1) Patch the holes - Look for loopholes in your original defect report that could be supported with further investigative data to strengthen your case. When you are going for an appeal, you should anticipate attacks on the weaker areas of your original report. You should understand that your report was weak and unpersuasive at the first place. So try and gather as mush information to make it appealing this time around.
2) Follow it up - Do some additional follow up testing around the original defect in an attempt to find more serious consequences. If you are able to find more areas that are affected by the defect and more severe consequences, it should add to your confidence level. A defect that infests a wider range of functionalities and has severe consequences has more chance of getting attention.
3) Follow the Money - There is a popular doctrine in criminal investigation; “in most of the crime cases, if you will follow the money you should soon able to reach the criminal”! Same can be applied in testing too, while appealing against rejection of a defect. Talk to the major stakeholders like the Managers, the Client, Sales department staffs, Technical Support team, and even the Technical Writers. Try to find out who will be most affected if the Product is shipped along with the defect. Try to get an idea of the financial loss that can result due to this defect if left unfixed. As James Bach defines – “A bug/defect is something that bugs someone who matters”! Try to identify the “someone” for whom your defect really matters and find out how costly it matters.
4) Build a Scenario to support your Testing – It’s time for story telling. This is where a tester’s story telling capability comes in handy. Use your imagination and your creativity to weave around a realistic story that sounds appealing and at the same time is capable of conveying the seriousness of the rejected defect. Build some scenarios that exemplify how a real-time user might come across the defect and how it might affect the user in a severe way.
5) Look out for similar defects in Competitors - Take advantage of the immense knowledge base of the Internet to find out some case where one of your competitors had released their Product with a defect similar to yours and had to face terrible consequences. Check in the recent press releases, forum discussions, news reports, courtroom cases for a similar case where a defect (similar to yours) had caused serious loss (financial loss in terms of loss in revenue, loss in credibility, loss in loyal customers etc) to a competitor. If you already take notes of important events related to testing, also look into your moleskine notebook for any similar incident that you might have recorded in there! If you are lucky enough to find such a case, your appeal should sound lot better in the review meeting!
With proper planning, you can increase the chance of the revival of your rejected defect in the review meeting. Do your homework well and build your case more effectively. But at any cost, never appeal for a defect that you yourself are not convinced with! If your further investigation reveals that this defect is not worth fighting for and has less chance of affecting an end user in real life scenario, better you should think twice before going for an appeal. Fighting for a defect that is not appealing to you, might result in waste of time and loss of your credibility as a software tester! Any thoughts?
Happy Testing…
This is excellent.I have some questions, can you please answer these questions?
ReplyDelete1.What is mean by out value in Testing ?
2.How to test the grass which is grown in a specific place?
3.What is the difference between end to end testing and system testing?
4.How do you implement a new testing process in a company which must replace existing process?
5.How will you do regression testing if you have more number of test cases..more than 1000 or 2000?
6.why do we need mandatory and assistive properties?
7.What is test effort?
8.why we need documents for testing?
9.Explain statement coverage?
10.How do we know about the build we are going to test? where do you see this?
11.How do you decide when you have 'tested enough'?
12.How can u test the white page?
13.Could anybody explain the difference b/w Patch, Build and Pilot?
Hi Debasis,
ReplyDeleteNice article on testing. Having being a software development manager before, I like your approach in putting a value to to bugs to determine how to handle it.
It is true, not every bug or defect is the same. I wish more developers will understand what you are stating here.
Great post!
Hi Lawrence,
ReplyDeleteI am glad that you liked my write-up. It is pleasing to hear these words from a person who himself has worked as a software development manager before. Thanks.
-Debasis