The Art of Bug Reporting: How to Market and Get Your Bugs Fixed?

Why there is a need for Marketing a Bug?

The first things that come to my mind as I start writing this article are the words of Cem Kaner – “The best tester isn’t the one who finds the most bugs or who embarrasses most programmers. The best tester is the one who gets most bugs fixed.”

Now – What is the difference between finding most bugs and getting most bugs fixed?

Isn’t it obvious that any bug logged in a bug management system should be fixed by the developer? The answer is No. Factors like time to market the product, time to complete the project on schedule and developers working in impractical tight schedules etc. force companies to release the product with few bugs which won’t affect the users largely.

[image source]

Who gives the confidence to the management m stating that the bugs present in the product won’t impact the customer’s confidence, reliability and the interest of stakeholders? – The Testing Engineer or the Testing team – it is the duty of each test engineer to get bugs fixed that might have the negative impact on product quality.

The priority of the bug, in my opinion, largely depends on how an issue is presented by the tester to the development and management teams.

Think of it like advertising or marketing the issue – this involves 2 steps:

1. Write or report bugs correctly
2. Know everything about the bug, so any further details can be explained better

The Art of Bug Reporting

Yes, bug reporting is an art. The way in which a bug is written shows the technical skill, domain expertise and communication capabilities of a test engineer.

Typically, a bug should contain the following information:

• Bug Summary
• Steps to reproduce
• Attachments (Snapshot, Log files etc.,)
• Bug reproducibility
• Bug Severity
• Software Version, Environment Information
• Other information based on organizational requirements.

An important note: Always dig deeper to find the root cause of the issue and report it. For instance, a simple login failure with the correct combination of username and password can be related to various reasons such as:

1. Login credentials not validated at all
2. Network timeout issues in case of remote logins
3. The system may consider all CAPS as non-CAPS.

So, as a Tester you should be able to decipher the differences while following the bug summary statements:

• “Unable to log in with the correct username and password”
• “Unable to log in with the correct username and password, when the username or password contains a mix of CAPS and non-CAPS alphabets.”

The latter is a very clear description of the problem and is unambiguous. With this, you not only increase your credibility as a tester but you are also reporting the actual problem instead of a symptom.

Now, let’s take a look at each field involved in a bug report and discuss the important aspects of each one:

#1. Bug Summary

A bug summary should provide a quick snapshot of what exactly the issue is. It has to be precise and well directed.

Example:

Apart from theory, I’ll try to explain with examples.

Let’s assume a simple login module. Let’s assume the issue as a new user visiting a website is unable to login with his default password. When I presented the same scenario to many of the students whom I trained during the initial phase of training, there were several answers as bug summary. Given below are few samples of how the summary looked like:

“New user is unable to login”

“User login is not working as expected”

“User is unable to login with correct password”

From the above samples, can you pick one statement that actually describes the problem? I don’t think so. The summary should always give a complete information about the failing scenario.

Consider the following statement:

“New user is unable to login with the default password provided via e-mail or an SMS”

As you can see, from the above statement a developer can clearly understand what the issue is and where the issue is.

So, try to find the right words to describe the summary that would give the information directly. Generic statements like “not working properly”, “not working as expected” etc., must be avoided.

#2. Steps to Reproduce and Attachments

Irreproducible bugs always take a back seat even though they may be significant. Therefore, take good care to write the steps correctly and descriptively.

Steps should be accurate and exactly the same as the ones that led to the problem. For functionality related bugs, the following sample is the best example.

Example:

Consider the same issue stated in the previous section.

1. Create a new user using Sign Up option in the home page. (Sample Username: HelloUser)
2. An e-mail and an SMS will be received with a default password. The e-mail ID and mobile number for SMS are provided while creating the user in Step #1. (Sample e-mail: HelloUser@hello.com, Sample Mobile Number: 444-222-1123)
3. Select Login option in the home page.
4. In the username text field, provide the Sample Username provided in Step #1.
5. In the password field, provide the default password received through an e-mail or SMS.
6. Click on Sign In button
7. Expected Result: User should be able to login with the provided username and password and navigate to the User Account Page.
8. Actual Result: “Invalid Username / Password” message is displayed.

If any of the information, is not provided in the above sample, then it will result in communication gaps and the developer will not be able to reproduce the issue. The steps have to be specific and detailed with the sample data that you use during testing.

If possible or wherever applicable, provide a snapshot of what you exactly see on the screen. This way, it will not only provide a good view of the issue to the developers but also an evidence of your test result.

The non-functional test cases such as stress, stability or performance test cases in addition to the above details, information about the scenario that causes the stress to the system can be reported as it is. Additionally, there are few systems which report logs for every operation that is performed. Logs are typically print statements provided by the developers in their code. Whenever a module is executed the corresponding logs will be printed or displayed. When logs are available, that would help developers to a great extent in reproducing the issue.

#3. Bug Reproducibility

An issue which is big or small will be prioritized based on the reproducibility. It can be seen always, sometimes, rarely or even only once. An issue that is reproduced as “always” will be prioritized higher than the rest.

So, it is the duty of a test engineer to track the scenario exactly for the issue that is always reproduced. Sometimes there might be few issues out of control of a test engineer which would result in an issue being reproduced only a few times but in multiple trials. In such cases, always mention the number of trials, a particular scenario is executed along with the number of times the issue is seen during those trials.

This, in turn, would add credibility to the bug report mentioned by you. Again, this would improve your reputation as a tester. I’d later tell you the reasons for having a good reputation.

#4. Bug Severity

Severity is undoubtedly one of the biggest influencers for prioritizing the Bug.

The following are the different categories of severity. Please note that these are just general samples and it varies from company to company.

1. Severity 1 – Show Stopper – for bugs that are catastrophic, without being fixed the user will not be able to continue using the software and there is no possible workaround
2. Severity 2 – High – for bugs similar to Severity 1, but there is a workaround
3. Severity 3 – Medium
4. Severity 4 – Low
5. Severity 5 – Trivial.

For example, let’s compare two similar issues.

In our set-top boxes, few service providers provide the frequency information of the service as currently tuned. Let’s assume that the frequency is displayed as 100 MHz instead of 100.20 MHz. This may not affect the user’s viewing of the services but may impact in terms of monitoring diagnostics of the set-tops. Hence this can be presented as a Severity 3 issue.

Assuming a similar issue in banking domain: If your account balance is displayed as $100, instead of $100.20, imagine the impact of the issue. This must be a Severity -1 defect. As you can see in both cases the issue is very similar that the UI is not displaying the digits after the decimal point. But, the impact varies according to the domain involved.

Efficient participation in Software Version Control Meetings

Usually, every organization has its own process to investigate and prioritize bugs. Generally, a meeting would occur at specific intervals during the project to discuss the bugs and prioritize the same.

The process during such meetings is as follows:

1. Query the list of bugs from the bug management system according to the severity.
2. Look at the summary and discuss the impact of the bug on the user’s experience on using a software product.
3. Based on the risk and impact assessment set the priority and assign the bug to an appropriate developer for fixing the same.

During step #2, it is imperative that every test engineer advocate the bug’s impact on the user experience if the bug doesn’t receive the priority it deserves. After all, it is we test engineers who consider the point of view of a user to write test cases and to test the product.

Consider the above example issue of not displaying the digits after decimal point in a banking domain. To a developer, it may seem like a less severe issue. He might argue that instead of declaring the variable as an integer, he’d declare that as a floating point to solve the issue and hence less severe.

But as a tester, your role is explaining customer’s situation. Your point should be how the user would complain in this scenario. The tester should say that this will cause panic among the users as the customer loses his money in cents.

Impact of not marketing a bug properly

If a bug is not marketed properly it will create issues like:

• Incorrect defect priority
• Delay in fixing the important issues
• Product release with severe defects
• Negative customer feedback
• Devaluing brand value

 

Apart from all the reasons mentioned above, it is very important to build your reputation as a test engineer. It is more like developing a brand value for your own self.

In the initial phase of your career, if you can keep your number of “Cannot Reproduce” or “Need More Info” or “Not a Valid Bug” or changes in severity as low as possible, at one stage your bugs will not be scrutinised at all and they’d be directly assigned to the appropriate developer to be fixed.

To develop such brand value and to win the confidence of your team and the development /or management teams, you’ve to develop some technical skills in terms of testing knowledge, domain and communication skills.

Conclusion

Any product or service either big or small is always bound to fail without proper advertising. Once a brand is established, any small product can be a super hit with the audience.

Having said that, over-advertising a product can also cause damage to the reputation.

So, a bug should always be written in a clear, concise and precise manner so that it gives an exact location of the bug in the extensive/exhaustive software map. I reiterate that this not only improves the quality of software but also reduces the cost of testing and developing the software to a great extent.

It’s not too late now! Let’s go ahead and get bugs fixed right away!