10 Questions to Ask When a Bug Report Is Created

Defect report, issue report, bug report. Whatever you end up calling it, the bug report happens to be a powerful tool in IT projects that can produce great results when put to effective use. So, how do you write an effective bug report? That is precisely the question we’re going to answer in this article.

We’re going to look at the 10 questions you need to ask when creating a bug report. These 10 questions will help you write a very good bug report. They will also help your colleagues to get the most out of reading your bug report through clear and actionable information.

Let’s jump right in and find out how you can write the gold standard of bug reports. Read more about bug report templates and examples

#1: Is that really a bug?

Yes – you’d be surprised how many times asking just this question could have saved many a tester from embarrassment.

“It isn’t the report itself, but the tester that writes it that makes a bug report effective.”

When you discover a bug, you need to validate it against something – ‘the requirement’. If the requirement doesn’t correlate with your definition of a bug, then there’s not much ground to stand on, is there?

Whilst ‘expected result’ and ‘actual result’ are good measures of defining what you’re testing for and what you observe in reality, ‘expected result’ needs to be based on hard, written requirements.

There are always exceptions to the rule

Testers, being gatekeepers for product quality, have the added burden of going above and beyond the requirement to identify bugs that don’t have requirements to back them up.

Minor examples would be when you identify formatting, copy errors or inconsistencies between two different pages of the journey.

Major examples could be like the gaping bug one of my testers successfully caught just a few days ahead of deploying a major internet banking upgrade:

As part of upgrading from a legacy internet banking system to a newer platform, my client commissioned a series of releases that moved legacy functionality page-by-page on to the new platform, providing customers a few new features every 6 to 8 weeks. One of the features that was moved was a customer information update page.

Functional requirements were written, user stories elaborated, legacy and new pages compared for gaps, test cases planned and signed off, executed, passed and test results accepted. Surprisingly, we achieved 100% pass rate for that particular release that included this feature as well – a feat never achieved before.

Then something happened. In the last few days leading up to release, one of my guys started exploratory testing the feature on a production server not exposed to customers yet. This phase is more of a health check to confirm everything works fine in production before you actually push it to a server that customers use.

The tester found that, while the feature worked as expected from a customer point of view, non-functional requirements covering exceptions that would be reported to a back-office operations team hadn’t been considered. This meant that while a customer could get a success message for any personal information update they initiated online, sometimes, due to a myriad number of reasons, an exception could be generated which refers the customer’s update request to the operations team.

“Of all things you can do right with a bug report, the one thing that you can do right the most is with its title.”

The operations team received such exceptions as an overnight report generated and sent to a team mailbox. There was only one problem – the report wasn’t getting generated any longer.

If you have ever worked with or for a bank in any capacity, you’d know by now this could result in a breach of SLA for responding to any customer requests, and expose the bank to reputational, regulatory and financial risks (among others).

Naturally, my guy brought this up, and we had a song and dance about whether the requirements state anything about back-office reports. The requirements obviously didn’t.

There was only one thing we could do – pull this particular feature from the release in the next few days. Thankfully, this was achieved by simply reverting to the legacy feature page link (we hadn’t demised it yet – not until the replacement was live and stable).

So, it’s okay to write a bug report that doesn’t necessarily relate to a specific requirement. Though you better have strong justification for it.

Pro tip: if you’re not sure, raise a ‘query’ instead of a bug. Most every testing tool in the market allows you to raise one, and then convert it to a bug if needed. This way, you save a lot of heartburn.

#2: Is the title/summary of the bug report crystal clear?

Now, of all things you can do right with a bug report, the one thing that you can do right the most is with its title.

They say that the ‘face is a window to the mind’. Similarly, the title/summary is the window through which everyone looks at your bug report. You want to nail your title for this precise reason.

How do you write the best title/summary for your bug report?

• • Cut out unwanted detail: any decent bug tracking tool will provide you a myriad set of fields and a bevy of customisations to allow you to enter a gazillion parameters that specify the system, component, function, feature, program, type of test, etc. So you don’t want to repeat all that information on your title, like a lot of testers are wont to do.
  • Keep it short and crisp: You can say a lot with a few words. Period.
  • Be specific: the summary needs to be powerful and clear to attract a developer’s attention. ‘UX not aligned properly’ isn’t something you would use to describe a high severity bug that warrants immediate action. There could be dozens more bugs which have the same problem, and hence, the same title. ‘Images not aligned with text as per design reference 42658’ is a major improvement. Being the most read part of a bug report, this title provides the reader enough information to understand the bug.

#3: What exactly is the bug?

Following on from the title/summary, the description and associated parameters on a bug report play a major role in ensuring the report is actionable.

Use the parameters to identify the system, component, program, feature, type of test, environment, channel or device used, version of operating system etc. These parameters help your team slice and dice the bugs list in any which way they need. Therefore, you should do the best possible job of filling out the parameters.

“The title is the most read part of a bug report.”

As a tester it is your responsibility to ensure the bug report is actionable. Bug parameters provide you a powerful way to achieve this.

The bug report has another way for you to provide the right level of information about your bug. Use the description field to explain how you found the bug, including:

• Steps to reproduce
• Expected result
• Actual result
• Any other relevant information.

#4: Is that a known bug?

Before you raise a bug report that others in your team will spend hours, even days to work on to resolve, check whether the bug has already been found. On projects that deliver successive releases, this is a reality, especially when you consider the same feature being tested repetitively across multiple releases.

It’s quite possible that a bug was discovered earlier, and is at some stage of being fixed. It’s also possible that the bug was discovered previously and de-prioritised as an isolated, infrequent issue.

Whatever the reason, it will be helpful to know if the bug you discovered has been found previously. If it were, then you have more options:

• Raise a bug report and link it to the original bug.
• If the bug has been found in a feature different to the original one, then establish that it is a recurring bug not limited to one system or component, and raise the bug profile.
• Where a fix is already under development, alert the developer to also cover this instance of the bug.

#5: Have you tested with the correct version of the software?

Too often, we find that testers have used the wrong version for a browser or tool to execute a test case. You can’t expect Chrome version 48 to continue to run your web page smoothly (we’re on version 63 now by the way).

“Whenever you see a defect that has remained unresolved past SLA, remember to ask whether the developer was able to reproduce the bug successfully at first go.”

Any bugs you find when using the wrong version of a browser or tool are, to put it mildly, invalid.

Good test managers work with their project teams to maintain a list of supported tools and versions. So, stay informed about the correct versions to use for any test cycle.

#6: How many bugs are you raising on this report (trick question)?

If the answer is ‘more than one’, you need to break down each bug and give it its own bug report.

There’s not a lot I need to say to explain the significance of this one. You need to ensure a bug report only reports a bug – not more.

#7: Can the developer reproduce the bug successfully?

This is a fundamental question, and one that pretty much determines if the resolution to this bug is going to be swift.

The bug report should contain all the relevant information a developer needs to reproduce the bug themselves. Which means you need to use everything we’ve discussed so far to produce a highly informative bug report that guides the developer step by step in recreating the bug from scratch.

Whenever you see a defect that has remained unresolved past SLA, remember to ask whether the developer was able to reproduce the bug successfully at first go.

I’ve seen too many developers and testers spend hours, if not days, exchanging communication to be able to get to the root cause of the bug. This time could have been easily put to better use had the tester spent the little additional effort required to ensure the developer can reproduce the bug easily.

#8: Have you assigned the right level of priority/severity?

Is the bug in question relegated to a customer journey sub-step that’s rarely used or triggered, or is it affecting more areas of the journey?

Is the bug blocking the entire function in question, or is it just cosmetic?

If we had to go live with the bug intact, what is the risk to the customer and our business?

How many test cases are blocked due to the bug?

Questions such as these will help you arrive at the right level of priority and severity for each bug. A bug report with the correct priority/severity assignments will go a long way to establish a ranked pipeline of defects for the project team to fix sequentially.

Note: I know a lot of testers that ‘play safe’ – by assigning everything as ‘high’, or worse still, ‘medium’. Too much effort is then spent in ranking and re-prioritising the flood of bug reports with the same priority/severity. Don’t fall into this trap.

#9: Are you presenting facts or opinion?

As a tester, this is one of the key questions you should be asking yourself when creating a bug report.

Why?

Sticking to facts will help you craft a clear and concise bug report. If you do have opinions on how the product should be designed or should have been built, well, keep that for Retrospectives and other feedback mechanisms designed to collect constructive inputs.

The bug report needs to focus on describing a verifiable issue with the sole purpose of fixing it. Nothing else.

#10: Are you writing a bug report or an indictment?

Perhaps you and the developer have never seen eye to eye on things. Perhaps the developer has rubbed you on the wrong side too many times. Perhaps the developer has overstepped boundaries and written you stinker emails on the most frivolous things.

Perhaps you don’t think much about the developer’s capabilities.

And, perhaps, you’re frothing at the corner of your mouth, waiting to get revenge for what you deem to be gross injustice.

I won’t pretend to know how it feels, or advise you on the best way to get back at this person.

What I’ll tell you is this: Don’t take it out on the developer through a bug report.

That is unprofessional, to say the least.

Whatever your pickle with an individual, your bug report should be impeccable and entirely based on fact. Any personal references or seeming inferences to the code quality or an individual, need to be kept out of the bug report.

Take any differences up directly with the individual or their manager – don’t use critical tools like the bug report to sting others.

Yes, we see it all too often when people use any medium at their disposal to settle scores.

A good tester can rise above such pettiness.

---

 

Now It’s Your Turn

A high-quality bug report is the true reflection of a tester’s capabilities. You should take pride in crafting an impeccable bug report every single time.

Remember, every time you capture that perfect, flawless bug report, a developer is silently thanking you for giving them a chance to fix a bug they may have introduced in the first place. And your team will thank you for making it easier to resolve bugs faster and deliver your product to customers sooner.

If you found this article helpful, how about sharing it with your friends through one of the sharing options at the top of this page?