Why there is a need for Marketing a Bug?
The first things that come to my mind as I start writing this article are the words of Cem Kaner – “The best tester isn’t the one who finds the most bugs or who embarrasses most programmers. The best tester is the one who gets most bugs fixed.”
Isn’t it obvious that any bug logged in a bug management system should be fixed by the developers? The answer is No. Factors like time to market the product, time to complete project on schedule and developers working in impractical tight schedules etc. force companies to release the product with few bugs which won’t affect the user largely.
Who gives the confidence to the management that the bugs present in the product won’t impact the customer’s confidence, reliability and the interest of stakeholders? – The Testing Engineer or Testing team – it is the duty of test engineer to get bugs fixed that might have negative impact on product quality.
The priority of the bug, in my opinion, largely depends on how an issue is presented by the tester to the development and management teams.
Think of it like advertising or marketing the issue – this involves 2 steps:
- Write or report bugs correctly
- Know everything about the bug so any further details can be explained better
What You Will Learn:
The Art of Bug Reporting
Yes, bug reporting is an art. The way in which a bug is written shows the technical skill, domain expertise and communication capabilities of a test engineer.
Typically, a bug should contain the following information:
- Bug Summary
- Steps to reproduce
- Attachments (Snapshot, Log files etc.,)
- Bug reproducibility
- Bug Severity
- Software Version, Environment Information
- Other information based on organizational requirements.
An important note: Always dig deeper to find the root cause of the issue and report it. For instance, a simple login failure with correct combination of username and password can be related to various reasons such as:
- Login credentials not validated at all
- Network timeout issues in case of remote logins
- The system may consider all CAPS as non-CAPS.
So, as a Tester you should be able to decipher the differences between following bug summary statements:
- “Unable to login with correct username and password”
- “Unable to login with correct username and password when the username or password contains a mix of CAPS and non-CAPS alphabets.”
The latter is clearly more clear, descriptive of the problem and unambiguous. With this, you’re not only increasing your credibility as a tester, but also reporting the actual problem instead of a symptom.
Let us now look at each field in a bug report and discuss the important aspect of each one:
#1. Bug Summary
A bug summary should provide a quick snapshot of what exactly the issue is. It has to be precise and well directed.
More than theory, I’ll try to explain with examples.
Let’s assume a simple login module. A new user visiting a website is unable to login with his default password is the issue. When I presented the same scenario to many of the students, I trained; during the initial phase of training, here are the few samples of what the summary looked like:
“New user is unable to login”
“User login is not working as expected”
“User is unable to login with correct password”
From the above samples, can you pick one statement that actually describes the problem? I don’t think so. The summary should always give complete information about the failing scenario.
Consider the following statement:
“New user is unable to login with default password provided via e-mail or SMS”
As you can see, from the above statement a developer can clearly understand what the issue is and where the issue is.
So, try to find the right words to describe the summary that gives the information straight away. Generic statements such as “not working properly”, “not working as expected” etc. must be avoided.
#2. Steps to Reproduce and Attachments
Irreproducible bugs always take a back seat even though they may be significant. Therefore, take good care as to write the steps correctly and descriptively.
Steps should be accurate and exactly the same as the ones that led to the problem. For functionality related bugs, the following sample can provide you an example.
Consider the issue stated in previous section.
- Create a new user using Sign Up option in the home page. (Sample Username: HelloUser)
- An e-mail and an SMS is received with a default password. The e-mail ID and mobile number for SMS is provided while creating the user in Step #1. (Sample e-mail: HelloUser@hello.com, Sample Mobile Number: 444-222-1123)
- Select Login option in the home page.
- In the username text field, provide the Sample Username provided in Step #1.
- In the password field, provide the default password received in e-mail or SMS.
- Click on Sign In button
- Expected Result: User should be able to login with the provided username and password and navigate to the User Account Page.
- Actual Result: “Invalid Username / Password” message is displayed.
Any of the information, if is not provided in the above sample will result in communication gaps and the developer will not be able to reproduce the issue. The steps have to be specific and detailed with sample data that you use during testing.
If possible or wherever applicable, provide snapshot of what you exactly see on the screen. This way, it not only provides a good view of the issue to the developers, but also an evidence to your test result.
In case of non-functional test cases such as stress, stability or performance test cases in addition to the above details, information about the scenario that causes the stress to the system shall be reported as it is. Additionally, there are few systems which report logs for every operation performed. Logs are typically print statements provided by developers in their code. Whenever a module is executed corresponding logs will be printed or displayed. If logs are available, that would help developers to a great extent in reproducing the issue.
#3. Bug Reproducibility
An issue big or small will be prioritized based on the reproducibility. It can be always, sometimes, rarely or only once seen. An issue that is reproduced “always” will be prioritized higher than the rest.
So, it is the duty of a test engineer to track the scenario exactly down to a level that the issue is always reproduced. Sometimes there might be few issues out of control of a test engineer which would result in an issue being reproduced only a few times in multiple trials. In such cases, always mention the number of trials a particular scenario is executed and number of times the issue is seen during those trials.
This would add credibility to the bug report mentioned by you. Again, this would improve your reputation as a tester. I’d later tell you the reason for having a good reputation.
#4. Bug Severity
Severity is undoubtedly one of the biggest influencers for prioritizing.
The following are the different categories of severity. Please note that these are just general samples and it varies for company to company.
- Severity 1 – Show Stopper – for bugs that are catastrophic, without being fixed the user will not be able to continue using the software and there is no possible work around
- Severity 2 – High – for bugs similar to Severity 1, but there is a work around
- Severity 3 – Medium
- Severity 4 – Low
- Severity 5 – Trivial.
For example, let’s compare two similar issues.
In our set-top boxes, few service providers provide the frequency information of the service currently tuned. Let’s assume that the frequency is displayed as 100 MHz instead of 100.20 MHz. This may not affect the user’s viewing of the services but may impact in terms of monitoring diagnostics of the set-tops. Hence this can be presented as a Severity 3 issue.
Assuming a similar issue in banking domain: If your account balance is displayed as $100, instead of $100.20, imagine the impact of the issue. This must be a Severity -1 defect. As you can see in both cases the issue is very similar that the UI is not displaying the digits after the decimal point. But, the impact varies according to the domain involved.
Efficient participation in Software Version Control Meetings
Usually, every organization has its own process to investigate and prioritize bugs. Generally a meeting would occur at specific intervals during the project to discuss the bugs and prioritize the same.
The process during such meetings is as follows:
- Query the list of bugs from the bug management system according to severity.
- Look at the summary and discuss the impact of the bug on the user’s experience on using software product.
- Based on the risk and impact assessment set the priority and assign the bug to appropriate developer for fixing the same.
During step #2, it is imperative that every test engineer advocate the bug’s impact on the user experience if the bug doesn’t receive the priority it deserves. After all, it is we test engineers who consider the point of view of a user to write test cases and to test the product.
Consider the example issue of not displaying the digits after decimal point in a banking domain. To a developer, it may seem like a less severe issue. He might argue that instead of declaring the variable as an integer, he’d declare that as a floating point to solve the issue and hence less severe.
But as a tester, your role is explaining customer’s situation. Your point should be how the user would complain in this scenario. The tester should say that, this will cause panic among the users as the customer loses his money in cents.
Impact of not marketing a bug properly
If a bug is not marketed properly it will create issues like:
- Incorrect defect priority
- Delay in fixing the important issues
- Product release with severe defects
- Negative customer feedback
- Devaluing brand value
Apart from all the reasons mentioned above it is very important to build your reputation as a test engineer. It is more like developing a brand value for you.
In the initial phase of your career, if you can keep your number of “Cannot Reproduce” or “Need more Info” or “Not a Valid Bug” or changes in severity as low as possible, at one stage your bugs will not be scrutinised at all and they’d be directly assigned to the appropriate developer to be fixed.
To develop such brand value and to win the confidence of your team and the development and/or management teams, you’ve to develop technical skills in terms of testing knowledge, domain and communication skills.
Any product or service, big or small, without proper advertising it is always bound to fail. Once a brand is established, any small product can be a super hit with the audience.
Having said that, over-advertising a product can also cause damage to the reputation.
So, a bug should always be written in a clear, concise and precise manner so that it gives an exact location of bug in the extensive/exhaustive software map. I reiterate that, this not only improves the quality of software but also reduces the cost of testing and developing the software to a great extent.
Okay Now! Let’s go and get bugs fixed right away!