Here comes my next blog. This time I am going to cover something about the Bug reporting process and guidelines which is very crucial for the testing team.

This paper is about what I’ve observed. I am not an empowered tester. But I am pretty good because I used to analyze the gifted people. I puzzle out what is that they do have and I don’t. I try to point out those things and then I do those things.

Now the first thing is "What is a Bug?” Bug is either a defect in the system under test (or device under test) or absence of support in the system under test for quality increasing features. It is a mismatch between the requirement specifications and the actual outcome. A behavior is viewed as a bug if the software doesn’t perform as required, specified or expected. Some synonyms are issue, fault, error, CR (Change Request), defect. I don't know why, but personally I like the word Bug.

First we have to understand the causes of a Bug in the software. In my total testing experience till date, I have observed some of the points that are mostly responsible for having bugs in the software. Those are:

1. Miscommunication or no communication between the teams
2. Software complexity
3. Lack of development experience
4. Poorly documented requirement specifications
5. Programming errors
6. Changing requirements on regular basis
7. Time pressures created by either management or client
8. Poorly documented code
9. Absence of any code writing guidelines
10. Poor guidance from development lead

Now the question is how a tester finds a bug? By executing the set of test cases, by monkey testing, by adhoc testing and many other ways. But personally I feel that it’s better to find bugs using the test cases rather than the monkey testing. Or vice versa if you are finding any bugs while monkey testing, add that test case on your test cases document. This will help you a lot. In one of my previous project, I was the person who had fired 75% of the total bugs in the system. But I was not updating the test cases. At last, when the product has been shipped to the customer, the test team has a very less number of test cases as compare to the bugs that have been logged. Although the management has this information that test team is not updating the test cases due to time crunch. But at last, we have to update all our test cases and shipped it to the customer.

We all know that when a tester finds a bug that should be communicated to the development team. A Bug Tracking system is the interface between the development team and the testing team to convey the bugs. It is constructed as a database. I have worked on three different types of Bug Tracking System till date. And all have more and less same guidelines to log a bug.

Now let's start. First thing is when an issue (referring issue here) is found by the tester, the first point is whether that is a bug or not? If that issue is mentioned in the release notes as a limitation or a known issue, it’s not a bug. Also if that issue is already defined in the requirements specifications, it is not a bug. In one of my previous project, I fired a bug with severity as showstopper (blocker) that was already specified as a limitation in the release notes. Instantly, that bug has been marked as Invalid. Although after that, development team lists out all the showstopper bugs as limitations in the next releases. Anyways, this should be taken care by the management.

Now comes the submission part of that bug. First you have to think about the synopsis/summary of the bug. It is very important, as this one line statement will describe the issue in short. A summary on the bug report communicates in a single sentence the essence and significance of the problem we have found. This is very important because when anyone (including your manager) goes through that bug, the summary should give a fare idea of what the problem is. A short and point-to-point summary will help everyone to understand the issue. And this summary is invaluable when it comes time to prioritize the bugs. I personally believe that writing this one line statement is much harder than it looks. Initially when I started testing and firing issues in Bug Tracking System, no one was able to understand what is being mentioned in the summary. To overcome this problem, what can be done is:

* Think about that issue twice that you have found.
* Make two or three summary statements for that
* Show all those summaries to your test lead.
* Ask him to select the appropriate summary for that bug.
* See what he is selecting and why?
* Discuss that summary line.

You have to do this exercise by not more than 6 to 7 times. After that you will be able to write a good summary. That's what I have done. Initially I spent much time to write good summaries because it is the most important sentence in the Bug report.

After summary, the next thing is the description part. Good bug reporting starts with organized and thorough testing. Testing may be automated or manual, but the tester should know what he is trying to achieve? The first thing that should be mentioned in the description is the release (or build) name or version number. The development and the testing team should do this. Although there is a drop down menu in all the Bug Tracking Systems to select the release version number, but this practice is useful for testing team when any comment is being added to that bug. Adding a comment with the release number helps to understand the flow of the bug life cycle with respect to releases. If you are logging in the Bug Tracking System using your company ID, the comment will be added with your company ID number. In this case, it is the responsibility of that person to specify his/her name at the end of the comment.

The tester should describe all the steps that have been performed to create that bug. If the bug is not reproducible, mention it. Try to recreate the scenario thrice. If that is repeatable all the three times, mention that the bug is reproducible. If the equation is something different, mention that also. Like "This failure occurred twice out of three tries". This is very important. Do remember:

If you repeat the scenario thrice or more than that, that will make your understanding clear about that issue and you have a better view for that. How the bug occurs deepens your understanding and helps convince developers and managers about the problem.

Try to dig the issues as deep as you can. I mean it is not a good practice that you have found a bug and you are mentioning the problem only. Try to dig the issue and see from where this problem is generating. If that is from the database, mention the table name and field name where the problem might exists. If that is in the code, try to debug it. If you are not able to dig, at least suggest some resolution to the development team. Its quite easy for the development team to understand and debug the issue if that is clearly written. Use just the words you need and describe only the steps and information that actually pertain to the bug being reported. Make sure that you are proving enough details so that development doesn't have to come looking for you to get them. In one of my previous project, I found a bug in which the application was not able to import a text file. When I dug a bit deeper, I found that the application is not able to import a specific kind of text file having special characters in it. Going more inside it, I found that the presence of only one special character in that text file is creating the problem. I have specified it in the description. And I hope that helped the developer to debug and fix the problem.

One more important thing is that before submitting the bug report in the Bug Tracking System, you should re-read that thoroughly so that if there is any thing left, you can add it. There is a short story of Mr. Perfect that I want to mention here.

Mr. Perfect is sending a love letter to his beauty. He wants to make sure it’s just right. So he doesn’t send it right away. He decided to wait a bit, so he pulls down new mail. One is from a major client. Mr. Perfect adds his address to his address book. He reads that mail and then after re-reading his love letter, he sends it. But unfortunately, the mail client, that he is using, sends the mail to the most recent person that has been added in its address book. Now you can imagine what happened?

So be very careful.

It’s a good practice to attach the screenshots and log files (if any) with the bug. Log files are useful for the development team to debug the problem. Screenshots are useful for development team as well as testing team. Sometimes when the problem is not consistent, the screenshot can show the exact picture of what have been done. In one of my previous project, I found a problem in the application only in the first attempt. The application was crashing giving a unique message that I have never seen before. After that I tried a lot to reproduce it but I couldn't do that. When I discussed that issue with the development team, they simply said it is not possible. And if yes, reproduce it, as they know that I was not able to reproduce it. But luckily I have the screenshot of that problem. I filed a bug with that screenshot as attachment and severity as showstopper, saying that the bug is not being reproducible. Till now, that bug is still in the Bug Tracking System as in open state, but the product is in the market.

Adding a comment in the existing bug is also very important. Sometimes a developer after fixing the problem adds a comment like "Fixed". This is not a good practice. The developer (or fixer) should mention where the problem was and what have been done to fix the issue. As a tester, adding a comment is also very important. I have worked with one of my colleague who used to write only "Verified and found fixed" while verifying the bug. But to keep track of what have been verified, the tester should mention like "Verified and found that .........................” This way the tester can mention the reason of closing the bug or reopening the bug.

Writing good bug reports and managing those reports is a challenging task for the test team especially for the testing manager. These challenges are both technical and political. Political here means related to the need to manage the interpersonal and team dynamics of criticizing the work of others. I once worked on a project where my lead was only concerned about the number of bugs that I have fired in the Bug Tracking System. In my opinion, quality of bugs is more important as compared to quantity of bugs. You should not enjoy testing just because you got a chance to "catch" developers. This kind of attitude will definitely create issues between the developers and the testers. We as a testing team should keep our heads cool during discussion about bug reports with developers. We should discuss bug reports with open mind, accepting the possibility that we are mistaken, but also defend the reasonable expectation of yours. As a tester you should be cooperative with the developers. I once worked in a technology, which was quite new to me at that time. I have to test the release and deliver it to the customer on weekly basis. At that time, I worked with one of the best development team and I realized that working with developers is quite useful for the testing team.

In software industry, Bug report ping-pong is a game played between developer and tester. Here in this game, bug reports are kicked unproductively back and forth. One of the most common reasons for this ping-pong game is that the developer says that this bug is not reproducible. This problem can be resolved through careful bug reporting particularly documenting the steps to reproduce and mentioning that the problem is reproducible or not. Second most common reason for this ping-pong game is inappropriate assignment of the bug to the developer. And the managers can resolve this by assigning the bug to the concerned developer. This article has hopefully challenged some of your existing processes of firing and managing the bug reports.

I have tried to provide an overview of a generic system, based on my experience. As you learn your company system, you may find that items and procedures are little different. But one thing I want to say that as a tester you should have strong opinions about the bug reporting and managing. Having this attitude in mind, you can definitely improve your bug reporting process and can handle the situation easily.

-- Sanat Sharma