Why testers need to be highly skillful at describing bugs

By 5th April 2013 Testing

This is Part 2 of Aleksis Tulonen’s guest post. Part 1, ‘What makes a bug, what makes the bug a problem and how do I handle it?‘ can be found here.

Bug reports are one of the main artifacts of our work and they also establish our credibility. Carelessly written bug reports (e.g. programmer can’t understand what the problem is, it’s hard to reproduce, there are a lot of grammar mistakes) give the impression that we are not professional testers.

In some cases bug reports are our main (and only) way of being in contact with the programmers. Because of these reasons I think that we need to be extremely good at describing bugs and in the end, at writing bug reports.

Personally, when writing a bug report, I focus my bug reports on information about configuration (e.g. browser, OS), providing steps for reproducing and attaching any files (e.g. screenshot, video, log files) that I think will help in reproducing and in the end fixing the bug.

However, I can’t spend too much time on the report. Even if I did, there’s always a risk that programmer will not be able to understand my bug report and will fail to reproduce the bug. I try to minimize this risk, but also at the same time I realize that the time I spend on writing the report is time away from finding new information.

 

I’ve seen too many examples in my career where there is a constant discussion between the tester and programmer regarding the additional information that the programmer needs in order to fix, reproduce or even understand the bug. Although not always the rule, often these moments can be handled by writing the bug report well enough in the first place.

 

That’s why I highly encourage people to do it professionally in the first place. Figure out as much (not too much) information as you can in reasonable time (requires judgment). Then tie the pieces together in a way that will maximize the quality of the information delivered. You can also ask for feedback from programmers, other testers or any of the stakeholders who handle your bug reports. Programmers could inform you if they feel that some information is missing from the reports. You could also ask for general comments from other testers regarding the overall quality of your bug reports.

 

Because I personally love examples and find them to be the most convenient way of understanding something, I’ll show an example of a bug report I wrote regarding a bug I found while using XMind (mind mapping tool). It’s not a model of a flawless bug report but rather an example of how I currently write bug reports in my project.

 

One of the reasons I wrote this post is that I want to make my work public. By publishing my thoughts and my example bug report I will get feedback that will help improve myself. That’s also one of the ways of becoming a software testing expert. Surround yourself with bright and diverse people who are not afraid to criticize your ideas. That way you get feedback that truly helps you become better at your work.

Bug report example – XMind

Before I show the bug report, I’ll let you see the error I saw when using XMind:

GP_RT_5-Why testers need to be highly skillful at describing bugs

Bug ID #1: User receives java.lang.NullPointerException when creating boundary for subtopic

 

Configuration

Windows Vista Enterprise – SP2 (32 bit)

 

XMind for Windows- v3.3.0

 

Summary

I received java.lang.NullPointerException –error when I created boundary two times in a row for subtopic, and then clicked left mouse button.

 

Steps to reproduce

  1. Select the [Central Topic]
  2. Create [Subtopic] with Tabulator or via menu
  3. Select the created [Main Topic 1]
  4. Press Ctrl+B+B
  5. Click the left mouse button when cursor is shaped as cross.

 

Expected

Selecting a subtopic, then clicking [Ctrl] down and two times [B] button, should not do anything when you press the [B] second time.

 

Actual

When I insert the boundary second time for subtopic and click left mouse button, when the cursor is shaped as cross, I get a java.lang.NullPointerException. I was not able to reveal any side effect, but user will feel confused, as s/he does not know if it has an effect on the behavior of XMind.

 

Attachments

I attached a screenshot of the error message I received and also a video that shows how I reproduced the problem.

 

In conclusion

As you’ve seen, finding and reporting bugs, is not a trivial matter. It involves various challenging phases and those can easily vary between contexts. I’m constantly trying to find the balance between the effort I put to polishing the information I provide, and on the other hand, to finding new information. Knowledge about oracles and tools can help you to improve the information you provide. In the end it’s though you, who will make the final decision regarding the information, that is included or not included.

 

Seek ways of getting feedback for your bug reports, or ask if you could give feedback to others. Sharpen your skills of observing, recognizing and describing problems. If you don’t know where to start from, I strongly recommend BBST Bug Advocacy course (requires that BBST Foundations is passed) that is provided by Association for Software Testing. I will personally participate that (BBST Bug Advocacy) on November this year.

 

Also, feel free to comment (either here or on Twitter) on my bug report or anything else that I’ve written on this article/post. Constructive feedback helps me move forward as a tester.

 

About the author

My name is Aleksis Tulonen and I’m a software tester. You can find me on Twitter (@al3ksis) and I also write my own blog at flowoftesting.wordpress.com. 3 years ago I started working as a system tester at a telecommunications company. Mainly my work was focused around routers, switches, and firewalls.

A year ago I became software test consultant at Comiq and since then I’ve been pursuing my goal to become a software testing expert. I have defined what this means for me, but if I ever reach it, I am going to redefine it. Other people can view me someday as an expert, but personally I will always be a student. This means that I’m studying testing constantly. That’s the path I have chosen. That’s also the path I enjoy travelling.

Comiq is a software testing and quality assurance service provider located in Helsinki, Finland. Our skillful staff has years of experience in testing and QA of business critical IT systems. Our company web page is located at www.comiq.fi.

Comiq was originally founded to serve businesses in electronic commerce and monetary transaction by providing specialized software testing solutions. Since then, the field of testing and quality assurance has developed fast, and today we offer software testing services independently of any particular technology or industry. Comiq was ranked on the Deloitte Technology Fast 50 listing in 2010, 2011 and 2012 and in the Great Place to Work list in Finland in 2011 and 2012.

Join the discussion 7 Comments

  • […] to figuring out if something is a problem (this relates to TCS quote above). In the end I’ll also show an example of what my bug report would look like. I will write a bug report of a bug I found on XMind (mind mapping tool – […]

  • […] initially got to know him on Twitter, but we got more familiar with each other, when he reviewed an article of mine. His review varied from others, as it included e.g. expressing emotions. I talked with him […]

  • Bryan says:

    This was an exceptionally good post. The author must have spent some time and made an actual effort to write a
    good article……..

  • Qazi says:

    Great article.
    All testers must read this article for their growth.

  • Kapurbirdy says:

    Reporting: Each team should have a agreement on the minimum information required to help them to understand and reproduce and fix a bug. Going above that is waste of time, effort ( money ).

    Evidence: As you have stated evidence ( video, sequence of screenshots etc ) is useful.

    Environment: very important to specify this. BUT if your environment is specified elsewhere and there are no deviations then reference to that specification including version number of the spec.

    Pre/Post test conditions: you need to capture this as there may be subtle differences that resulted in the bug being revealed.

    intermittent bugs ( ones you cannot reproduce at will): capture as much as you can and report the issue. A number of such occurrences and the information captured will eventually uncover the pattern. Usually a certain sequence of events, status of dynamics result in the bug. MY personal experience is that you start by capturing high level details. As you analyse and determine the possible cause or area effected then capture more info on that area. Eventually sufficient details reveal the pattern of sequence of events, data and preconditions that trigger the bug conditions.

    MOTTO – You cannot fix something that you cannot understand.

    Does this help?

  • Paul Mason says:

    I’m pretty much in complete agreement, I’d add a couple of thoughts… that any bug report should also help to get an actual conversation going (IM, Google Hangouts, Skype, or perhaps even in real life!) and build up a rapport between your team members, and; definitely write out some steps to help replicate the bug but don’t go overboard by nailing things down to the nth degree as there may be more problems lurking that could be missed by the rush to fix that 1 thing! Just my 2p 🙂

  • […] experienced. Similarly, there is a difference in the details and description of the reported bug. Experienced testers know what information is required and useful to developers. On the other hand, novice tester may rely on writing a superficial description without much clue […]

Leave a Reply