The Best Ways to Utilize Bug Report Templates

Unstructured bug report data is difficult to use. You know that and I know that. That’s why bug report templates are a critical tool in organizing the data you receive from users. Of course, a bug report template on its own is not enough. You will find out everything you ever wanted to know about bug report template but were afraid to ask in today’s article. Well, maybe not quite everything…

Understand The Big Picture Of Your Bugs

Reading a single bug report gives you an understanding of a specific problem. Using standard bug report template makes it easier to understand the big picture. That matters for a few reasons – improved diagnosis and management reporting. Let’s go point by point.

Making the correct diagnosis for a software bug is critical to making the most of your efforts. Let’s explain the problem by adapting a famous analogy about production problems in a factory.

Find Out The Real Problem With Improved Bug Diagnosis

Picture this: you are working as the manager of a large facility producing paint for different clients. Maintaining high quality standards in the paint you ship is critical to keeping your customers.

Recently, you have received complaints that some of your paint has been watered down. You call in a few staff to discuss the problem. Each person presents a different “bug report.” One person points out that your front-line staff are tired from putting in long hours. Another person points out that the assembly line has been getting slippery. Finally, your maintenance manager comments that the daily cleaning process is taking longer than usual. What ended up as the ultimate cause of these bug reports? A water pipe over the production floor was leaking water into the paint! That’s an example of the problems you can detect by analyzing multiple bug reports.

There’s just one problem with the story above. It is difficult for the manager to sort through the conflicting opinions and find the underlying cause. Fortunately, we can make that assessment easier by adding structured elements to the bug report template. For instance, you may have fields that record a user’s web browser and version, operating system, connectivity status and a free text field for comments.

Identify trends in your bug reports over time

Over time, you may receive hundreds or thousands of bug reports. If you use bug report templates, you then have the ability to detect significant trends. For example, you might summarize bug report data in a PivotTable report in Excel. That’s a reasonable approach if you are getting started and have not yet discovered the right approach.

Here are a few items to look in trending bug reports over time:

Quantity. Start with the simple question – are the absolute number of software bugs increasing over time? Consider setting green (e.g. number of bugs reported is within 5% of the past period), yellow (e.g. 10% increase in bugs vs past month/quarter), and red (e.g. number of bugs reported have increased more than 10% vs past period) to summarize this measure.

For more insight on assessing the quantity of bugs, read How Many Software Bugs Are Too Many?

Strategic relevance. If your company is phasing out support for a given product, you may want to weight software bugs reported against that product accordingly.
Qualitative Trend. Assuming you have a free text field in your bug report template, consider running analysis against this data. Do users leave the field blank? Or do they use it to voice their complaints? Either way, there are insights to be mined from this section.
Customer weight. As we know from the 80/20 principle, some customers are worth more than others. Keep that principle in mind by applying more “weight” to bug reports that come from high value customers.

Understanding your bugs over time helps you provide appropriate guidance to management. Correctly diagnosing the problem (i.e. seeing the forest instead of the trees) helps you to focus your effort on the right problem.

Once you understand the problem, how to get these software bugs fixed, you can then start assigning the right people to the job.

To Fix Or Not To Fix – That Is The Quest

In the course of building complex software, you are bound to encounter a steady stream of bug reports. Deciding which software bugs deserve immediate attention from your team is an important product and management decision. Bug report templates make the decision-making process easier.

Let’s unpack the process further.

Assess The Bug’s Significance With Bug Report Templates

Less experienced software professionals panic when something unexpected happens to their code. The rest of us know the truth – surprises happen! As we explained in 10 Questions to Ask When a Bug Report Is Created, “When you discover a bug, you need to validate it against something: ‘the requirement’.”

For example, a bug report indicating integration problems with Microsoft Outlook may not matter if 95% of your customers use G Suite.

Here are three ways that bug report templates help you identify real problems:

Known Bug. As your team expands and more users interact with your product, duplicate bug reports are bound to occur. Consider including a field in your bug report template to direct the user to check your database of known bugs before creating another report.
Duplicate The Bug. Some software bugs are difficult or impossible to duplicate. In those cases, there will be a struggle to solve the problem. Consider this factor when evaluating the work effort involved in solving the bug.
Self-Assessment of Bug Severity. Ask the person filing the bug report to self-assess the bug’s importance. To simplify matters, use a simple four-point scale (e.g. 1 is “low importance” and 4 is “high importance). High importance bugs will then merit a closer examination.

What Is The Value Added By Solving The Bug?

Many people in development and engineering are passionate about quality. This admirable quality sometimes crosses the line into perfectionism. To protect your time against this challenge, step back and think about the value added by solving a bug.

I recommend breaking the value added concept into three points: technical cost/benefit, strategic relevance and customer benefit. Ideally, include a short section on each of these points on the bug report template.

Technical Cost and Benefits

Ask your developers to estimate the technical cost and effort required to solve the bug on a four-point scale. If they estimate it will take a few days to solve a problem and you have a launch date to meet, the decision to defer action on the bug is easier. On the other hand, the benefits of an improvement – patching a critical security problem – may be so high it makes sense to expedite that fix.

The Strategic Value Of The Bug

Consider the following scenario – your business development team has just signed a deal with another firm. As a result, your API suddenly became critically important to achieving an effective partnership. As a result, there is value in adjusting your bug report template to identify whether a bug involves strategic functions.

Customer Satisfaction

If a bug directly impacts customers, that factor needs to be considered. Collecting a yes or no response to “does this bug impact customers?” is a reasonable starting point. You might also add a further field to collect data on the customer’s account type (e.g. an enterprise user with 100,000 records and a personal user). You might also connect your bug report to customer support files to see if the customer has frequently complained about the problem.

What if your approach to bug reporting and tracking is already well established? Is there anything more you can do to leverage this resource? Yes – you can make better management decisions.

Ground Management Decision In Reality

Clueless managers have been mocked endlessly in Dilbert comics, blogs and elsewhere for a good reason. Poorly informed management decisions do real harm in the form of lost customers, demoralized staff or losing ground to customers. By applying analytics to your bug report templates, you can make better management decisions. Let’s look at two specific ways you can use manager better: annual planning and staffing decisions.

Annual Planning

Drawing up annual goals and plans for your department should draw on both data and desire. On the data front, having clarity on what has been achieved this year informs you on what you can achieve next. Bug reports also inform the “desire” aspect: what problems do customers keep raising? What problems are taking the most time for development and customer support to address properly? Using bug report data to formulate your annual goals means that you will avoid chasing unpractical or irrelevant aims.

Staffing: Annual Reviews & Hiring

Is your company overstretched in terms of software testing and quality? Analyzing data from your bug reports is one way to get at this point. If issues are repeated and staff appear unable to address them due to other priorities, you may need to hire additional staff. Pointing to a list of open bug reports is an excellent visual way of demonstrating the need for additional employees.

Bug reports also play a role in annual reviews for staff. Before, you groan and click away, bear with me for a moment. When you work in quality assurance, testing or a similar function, you may have some difficulties demonstrating the value you achieve. You cannot point to sales numbers. However, you can demonstrate improved product quality and reliability by looking at bug reports. Specifically, analysis of bug report templates will inform whether quality is improving over time.

Over To You

What goes into your bug report templates for your software? How do you make use of that information to deliver improvements over time?