The Art of Bug Reporting: How to Market and Get Your Bugs Fixed?

Why there is a need for Marketing a Bug?

The first things that come to my mind as I start writing this article are the words of Cem Kaner“The best tester isn’t the one who finds the most bugs or who embarrasses most programmers. The best tester is the one who gets most bugs fixed.”

Now – What is the difference between finding most bugs and getting most bugs fixed?

Isn’t it obvious that any bug logged in a bug management system should be fixed by the developers? The answer is No. Factors like time to market the product, time to complete project on schedule and developers working in impractical tight schedules etc. force companies to release the product with few bugs which won’t affect the user largely.

Bug reporting art

[image source]

Who gives the confidence to the management that the bugs present in the product won’t impact the customer’s confidence, reliability and the interest of stakeholders? – The Testing Engineer or Testing team – it is the duty of test engineer to get bugs fixed that might have negative impact on product quality.

The priority of the bug, in my opinion, largely depends on how an issue is presented by the tester to the development and management teams.

Think of it like advertising or marketing the issue – this involves 2 steps:

  1. Write or report bugs correctly
  2. Know everything about the bug so any further details can be explained better

The Art of Bug Reporting

Yes, bug reporting is an art. The way in which a bug is written shows the technical skill, domain expertise and communication capabilities of a test engineer.

Typically, a bug should contain the following information:

  • Bug Summary
  • Steps to reproduce
  • Attachments (Snapshot, Log files etc.,)
  • Bug reproducibility
  • Bug Severity
  • Software Version, Environment Information
  • Other information based on organizational requirements.

An important note: Always dig deeper to find the root cause of the issue and report it. For instance, a simple login failure with correct combination of username and password can be related to various reasons such as:

  1. Login credentials not validated at all
  2. Network timeout issues in case of remote logins
  3. The system may consider all CAPS as non-CAPS.

So, as a Tester you should be able to decipher the differences between following bug summary statements:

  • “Unable to login with correct username and password”
  • “Unable to login with correct username and password when the username or password contains a mix of CAPS and non-CAPS alphabets.”

The latter is clearly more clear, descriptive of the problem and unambiguous. With this, you’re not only increasing your credibility as a tester, but also reporting the actual problem instead of a symptom.

Let us now look at each field in a bug report and discuss the important aspect of each one:

#1. Bug Summary

A bug summary should provide a quick snapshot of what exactly the issue is. It has to be precise and well directed.

Example:

More than theory, I’ll try to explain with examples.

Let’s assume a simple login module. A new user visiting a website is unable to login with his default password is the issue. When I presented the same scenario to many of the students, I trained; during the initial phase of training, here are the few samples of what the summary looked like:

New user is unable to login”

“User login is not working as expected”

“User is unable to login with correct password”

bug summary wrong and right

From the above samples, can you pick one statement that actually describes the problem? I don’t think so. The summary should always give complete information about the failing scenario.

Consider the following statement:

“New user is unable to login with default password provided via e-mail or SMS”

As you can see, from the above statement a developer can clearly understand what the issue is and where the issue is.

So, try to find the right words to describe the summary that gives the information straight away. Generic statements such as “not working properly”, “not working as expected” etc. must be avoided.

#2. Steps to Reproduce and Attachments

Irreproducible bugs always take a back seat even though they may be significant. Therefore, take good care as to write the steps correctly and descriptively.

Steps should be accurate and exactly the same as the ones that led to the problem. For functionality related bugs, the following sample can provide you an example.

Example:

Consider the issue stated in previous section.

  1. Create a new user using Sign Up option in the home page. (Sample Username: HelloUser)
  2. An e-mail and an SMS is received with a default password. The e-mail ID and mobile number for SMS is provided while creating the user in Step #1. (Sample e-mail: HelloUser@hello.com, Sample Mobile Number: 444-222-1123)
  3. Select Login option in the home page.
  4. In the username text field, provide the Sample Username provided in Step #1.
  5. In the password field, provide the default password received in e-mail or SMS.
  6. Click on Sign In button
  7. Expected Result: User should be able to login with the provided username and password and navigate to the User Account Page.
  8. Actual Result: “Invalid Username / Password” message is displayed.

defect report steps

------------

Any of the information, if is not provided in the above sample will result in communication gaps and the developer will not be able to reproduce the issue. The steps have to be specific and detailed with sample data that you use during testing.

If possible or wherever applicable, provide snapshot of what you exactly see on the screen. This way, it not only provides a good view of the issue to the developers, but also an evidence to your test result.

In case of non-functional test cases such as stress, stability or performance test cases in addition to the above details, information about the scenario that causes the stress to the system shall be reported as it is. Additionally, there are few systems which report logs for every operation performed. Logs are typically print statements provided by developers in their code. Whenever a module is executed corresponding logs will be printed or displayed. If logs are available, that would help developers to a great extent in reproducing the issue.

#3. Bug Reproducibility

An issue big or small will be prioritized based on the reproducibility. It can be always, sometimes, rarely or only once seen. An issue that is reproduced “always” will be prioritized higher than the rest.

cant reproduce a defect

So, it is the duty of a test engineer to track the scenario exactly down to a level that the issue is always reproduced. Sometimes there might be few issues out of control of a test engineer which would result in an issue being reproduced only a few times in multiple trials. In such cases, always mention the number of trials a particular scenario is executed and number of times the issue is seen during those trials.

This would add credibility to the bug report mentioned by you. Again, this would improve your reputation as a tester. I’d later tell you the reason for having a good reputation.

#4. Bug Severity

Severity is undoubtedly one of the biggest influencers for prioritizing.

The following are the different categories of severity. Please note that these are just general samples and it varies for company to company.

  1. Severity 1 – Show Stopper – for bugs that are catastrophic, without being fixed the user will not be able to continue using the software and there is no possible work around
  2. Severity 2 – High – for bugs similar to Severity 1, but there is a work around
  3. Severity 3 – Medium
  4. Severity 4 – Low
  5. Severity 5 – Trivial.

For example, let’s compare two similar issues.

In our set-top boxes, few service providers provide the frequency information of the service currently tuned. Let’s assume that the frequency is displayed as 100 MHz instead of 100.20 MHz. This may not affect the user’s viewing of the services but may impact in terms of monitoring diagnostics of the set-tops. Hence this can be presented as a Severity 3 issue.

Assuming a similar issue in banking domain: If your account balance is displayed as $100, instead of $100.20, imagine the impact of the issue. This must be a Severity -1 defect. As you can see in both cases the issue is very similar that the UI is not displaying the digits after the decimal point. But, the impact varies according to the domain involved.

Efficient participation in Software Version Control Meetings

Usually, every organization has its own process to investigate and prioritize bugs. Generally a meeting would occur at specific intervals during the project to discuss the bugs and prioritize the same.

The process during such meetings is as follows:

  1. Query the list of bugs from the bug management system according to severity.
  2. Look at the summary and discuss the impact of the bug on the user’s experience on using software product.
  3. Based on the risk and impact assessment set the priority and assign the bug to appropriate developer for fixing the same.

During step #2, it is imperative that every test engineer advocate the bug’s impact on the user experience if the bug doesn’t receive the priority it deserves. After all, it is we test engineers who consider the point of view of a user to write test cases and to test the product.

Consider the example issue of not displaying the digits after decimal point in a banking domain. To a developer, it may seem like a less severe issue. He might argue that instead of declaring the variable as an integer, he’d declare that as a floating point to solve the issue and hence less severe.

But as a tester, your role is explaining customer’s situation. Your point should be how the user would complain in this scenario. The tester should say that, this will cause panic among the users as the customer loses his money in cents.

Impact of not marketing a bug properly

If a bug is not marketed properly it will create issues like:

  • Incorrect defect priority
  • Delay in fixing the important issues
  • Product release with severe defects
  • Negative customer feedback
  • Devaluing brand value

 Impact of not marketing a bug properly

Apart from all the reasons mentioned above it is very important to build your reputation as a test engineer. It is more like developing a brand value for you.

In the initial phase of your career, if you can keep your number of “Cannot Reproduce” or “Need more Info” or “Not a Valid Bug” or changes in severity as low as possible, at one stage your bugs will not be scrutinised at all and they’d be directly assigned to the appropriate developer to be fixed.

To develop such brand value and to win the confidence of your team and the development and/or management teams, you’ve to develop technical skills in terms of testing knowledge, domain and communication skills.

Conclusion

Any product or service, big or small, without proper advertising it is always bound to fail. Once a brand is established, any small product can be a super hit with the audience.

Having said that, over-advertising a product can also cause damage to the reputation.

So, a bug should always be written in a clear, concise and precise manner so that it gives an exact location of bug in the extensive/exhaustive software map. I reiterate that, this not only improves the quality of software but also reduces the cost of testing and developing the software to a great extent.

Okay Now! Let’s go and get bugs fixed right away!

Recommended reading

13 comments ↓

#1 NagiReddy

Nice article. Useful points.

#2 Srijeet

bug reporting is really a skill every tester needs to learn. it will break yur reputation in the team if you max bugs are getting rejected

#3 prashant

thanks for, usefull article

#4 Phani

thanks very good article

#5 Manikandan.K

Nice article

#6 Ritesh(Ritz)

Thank you so much for such an innovative article

#7 Karabo

The article is absolutely correct. But for us consultants who sit in an already established organisation like banks we struggle to practice some of the tips on this article. All because we aim at satisfying business and their timelines, meaning business will call the shots when it comes to defect priority and severity. If it was up to me, to market a defect would be easy, but sitting in a bank were we have ongoing changes as a result of production issues, chances are you will find defects but wont get most of them fixed no matter how you market it. We need to also be able to adapt

#8 Content Author

@nagireddy, @prashant, @phani, @Manikandan, @Ritesh – Thanks a lot for your valuable comments.

@Srijeet – Thanks for the comment. You’re absolutely Correct

@Karabo – Thanks for the comment. In most of the organisations, the situation that you say exists. As I’ve mentioned in the article, apart from how serious the bug is, it is the business decision that is going to call the shots. The point I am trying to make here is, inside a scope of particular business decision, the tester indeed has to market the bugs so that the bugs get fixed on time. We have to help the management understand that, the business decision will make sense only of this bug gets fixed.

#9 Joe DeMeyer

Greetings!

Great article! I have a few opinions to share.

Duty
I believe framing bug resolution as “…the duty of test engineer…” feels a little strong. When I write bug reports or coach testers to write bug reports, I prefer to advocate for its resolution by providing enough information to assist in the decision to fix the bug. In a similar manner, this helps frame the impact on quality.
I prefer testers be involved at just two places in the life cycle of a bug report: when it is created, and when it is ready for re-test. Everything else feels like project management.

Priority
In my opinion, it is never the responsibility of the testing team to assign the priority of a bug. Testing can provide an opinion, or they can clarify the impact to assist in assigning the priority.

Bug Report
I recommend including the bug’s impact on business process or business value in the bug report. This demonstrates the tester’s understanding of the business and frames the impact in terms that business team members understand. It makes it a little easier to sell.

I appreciate the awesome example bug report you provided. The detail is excellent. I also think the consequences you list are very relevant. Are you a graduate of the BBST Bug Advocacy course?

Lastly, I like your thoughts on building a reputation. This is often ignored by testers and it has such a large impact because testing is very social. I’d like to hear more on your thoughts about it!

Joe

#10 Content Author

@Joe:
Thanks for detailed feedback which is very categorical.

I completely agree with your comments.
But my points on “Duty” and “Priority” – I have been working on software teams of various sizes. Probably my points on those could’ve been influenced by very smaller teams who are also involved in Project management along with SQC as well.
For large organisations, your comments are indeed true.

And, I am not a graduate of BBsT Bug Advocacy course. Every point that I’ve mentioned is completely from my experience in Software testing. Sure, I’ll plan to write a separate article on the same.

#11 Content Author

@Joe:
Thanks for detailed feedback which is very categorical.

I completely agree with your comments.
But my points on “Duty” and “Priority” – I have been working on software teams of various sizes. Probably my points on those could’ve been influenced by very smaller teams who are also involved in Project management along with SQC as well.
For large organisations, your comments are indeed true.

And, I am not a graduate of BBsT Bug Advocacy course. Every point that I’ve mentioned is completely from my experience in Software testing. Sure, I’ll plan to write a separate article on the same.

#12 kaushik

I love this website . There’s an abundance of knowledge here.

Thank you STH team.

#13 Anurag

wow , such a detailed analysis.
STH team doing fabulous job .

Thank you .

Leave a Comment